Jul 10 18:17:20.453088 ip-10-0-132-151 systemd[1]: Starting Kubernetes Kubelet... Jul 10 18:17:20.953902 ip-10-0-132-151 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:17:20.953902 ip-10-0-132-151 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 18:17:20.953902 ip-10-0-132-151 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:17:20.953902 ip-10-0-132-151 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 18:17:20.953902 ip-10-0-132-151 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:17:20.953902 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.948396 2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955181 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955199 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955203 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955207 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955210 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955213 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955217 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955220 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955223 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955226 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955228 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955231 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955234 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955237 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955240 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955243 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955246 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955248 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:17:20.955756 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955251 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955254 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955257 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955264 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955267 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955270 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955275 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955281 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955284 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955287 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955292 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955296 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955299 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955302 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955306 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955309 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955312 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955314 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955318 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:17:20.956659 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955320 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955324 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955327 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955330 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955333 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955335 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955338 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955341 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955344 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955347 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955350 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955354 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955357 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955360 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955363 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955366 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955368 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955371 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955375 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955378 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:17:20.961560 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955381 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955384 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955387 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955391 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955394 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955396 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955400 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955404 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955408 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955412 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955415 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955418 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955421 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955424 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955428 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955431 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955434 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955438 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955441 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:17:20.962273 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955443 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955446 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955449 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955451 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955455 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955458 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955461 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955464 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955467 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955470 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955473 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955476 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955479 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955482 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955485 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955488 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955490 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955493 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955496 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955498 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:17:20.963009 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955501 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955504 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.955507 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956319 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956326 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956329 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956332 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956335 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956339 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956341 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956344 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956347 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956350 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956353 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956356 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956359 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956362 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956367 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956370 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:17:20.964680 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956372 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956375 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956378 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956381 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956385 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956388 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956391 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956394 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956397 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956400 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956403 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956406 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956408 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956411 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956414 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956416 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956419 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956422 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956427 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:17:20.965413 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956431 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956434 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956437 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956441 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956446 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956450 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956454 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956457 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956460 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956463 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956467 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956470 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956474 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956477 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956480 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956484 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956487 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956490 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956493 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:17:20.966121 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956496 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956498 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956501 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956505 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956507 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956511 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956514 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956517 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956520 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956522 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956525 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956528 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956531 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956533 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956536 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956539 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956541 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956544 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956547 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956550 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:17:20.968518 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956552 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956555 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956558 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956560 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956563 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956566 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956569 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956572 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956575 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956578 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956581 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956583 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956586 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956589 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956591 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956594 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956597 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956600 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956603 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956605 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:17:20.969484 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956608 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956610 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956613 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956616 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.956618 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957079 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957085 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957089 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957092 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957096 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957099 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957102 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957105 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957109 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957112 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957115 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957118 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957120 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957124 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957127 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:17:20.970879 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957130 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957134 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957138 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957141 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957144 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957147 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957150 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957153 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957156 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957158 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957161 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957164 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957183 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957186 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957189 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957192 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957195 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957198 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957201 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:17:20.971585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957204 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957206 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957209 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957212 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957214 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957218 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957221 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957227 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957229 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957232 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957235 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957238 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957241 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957244 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957247 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957250 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957254 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957257 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957261 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:17:20.973472 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957264 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957267 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957270 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957273 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957276 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957279 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957282 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957284 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957287 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957290 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957292 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957295 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957298 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957301 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957303 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957306 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957309 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957312 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957314 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957317 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:17:20.974158 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957321 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957327 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957330 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957333 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957336 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957339 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957344 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957347 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957349 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957352 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957355 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957381 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957385 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957389 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957394 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957398 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957400 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957403 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957406 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957409 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:17:20.974873 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957412 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957414 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957417 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957420 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957423 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.957425 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957511 2574 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957522 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957533 2574 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957542 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957549 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957557 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957566 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957573 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957579 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957584 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957588 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957592 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957595 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957600 2574 flags.go:64] FLAG: --cgroup-root="" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957603 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957606 2574 flags.go:64] FLAG: --client-ca-file="" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957609 2574 flags.go:64] FLAG: --cloud-provider="external" Jul 10 18:17:20.975585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957612 2574 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957617 2574 flags.go:64] FLAG: --cluster-domain="" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957620 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957623 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957626 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957629 2574 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957641 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957645 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957648 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957652 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957655 2574 flags.go:64] FLAG: --contention-profiling="false" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957658 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957661 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957665 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957668 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957672 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957676 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957679 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957682 2574 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957685 2574 flags.go:64] FLAG: --enable-server="true" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957688 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957695 2574 flags.go:64] FLAG: --event-burst="100" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957699 2574 flags.go:64] FLAG: --event-qps="50" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957702 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 18:17:20.976294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957705 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957708 2574 flags.go:64] FLAG: --eviction-hard="" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957712 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957716 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957719 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957723 2574 flags.go:64] FLAG: --eviction-soft="" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957726 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957730 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957733 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957736 2574 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957739 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957742 2574 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957746 2574 flags.go:64] FLAG: --feature-gates="" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957750 2574 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957755 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957758 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957761 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957765 2574 flags.go:64] FLAG: --healthz-port="10248" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957768 2574 flags.go:64] FLAG: --help="false" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957771 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-132-151.ec2.internal" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957775 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957778 2574 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957781 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957784 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 18:17:20.977449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957788 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957791 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957794 2574 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957797 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957800 2574 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957805 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957808 2574 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957811 2574 flags.go:64] FLAG: --kube-reserved="" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957814 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957818 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957821 2574 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957824 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957827 2574 flags.go:64] FLAG: --lock-file="" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957830 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957834 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957837 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957843 2574 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957846 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957849 2574 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957852 2574 flags.go:64] FLAG: --logging-format="text" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957855 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957859 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957864 2574 flags.go:64] FLAG: --manifest-url="" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957867 2574 flags.go:64] FLAG: --manifest-url-header="" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957871 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 18:17:20.978163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957875 2574 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957880 2574 flags.go:64] FLAG: --max-pods="110" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957883 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957886 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957889 2574 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957892 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957895 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957899 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957902 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957910 2574 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957913 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957916 2574 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957921 2574 flags.go:64] FLAG: --pod-cidr="" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957924 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957930 2574 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957933 2574 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957936 2574 flags.go:64] FLAG: --pods-per-core="0" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957940 2574 flags.go:64] FLAG: --port="10250" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957943 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957946 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06d0511a7001b4a75" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957949 2574 flags.go:64] FLAG: --qos-reserved="" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957953 2574 flags.go:64] FLAG: --read-only-port="10255" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957957 2574 flags.go:64] FLAG: --register-node="true" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957960 2574 flags.go:64] FLAG: --register-with-taints="" Jul 10 18:17:20.979101 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957964 2574 flags.go:64] FLAG: --registry-burst="10" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957967 2574 flags.go:64] FLAG: --registry-qps="5" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957970 2574 flags.go:64] FLAG: --reserved-cpus="" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957973 2574 flags.go:64] FLAG: --reserved-memory="" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957978 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957983 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957986 2574 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957989 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957992 2574 flags.go:64] FLAG: --runonce="false" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957995 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.957998 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958001 2574 flags.go:64] FLAG: --seccomp-default="false" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958004 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958007 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958010 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958014 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958017 2574 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958020 2574 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958023 2574 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958027 2574 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958030 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958035 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958038 2574 flags.go:64] FLAG: --system-cgroups="" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958041 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958047 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 18:17:20.979815 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958050 2574 flags.go:64] FLAG: --tls-cert-file="" Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958053 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958057 2574 flags.go:64] FLAG: --tls-min-version="" Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958060 2574 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958063 2574 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958066 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958070 2574 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958073 2574 flags.go:64] FLAG: --v="2" Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958078 2574 flags.go:64] FLAG: --version="false" Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958082 2574 flags.go:64] FLAG: --vmodule="" Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958086 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.958090 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958260 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958269 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958273 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958276 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958279 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958281 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958285 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958288 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958290 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958293 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:17:20.980749 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958296 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958299 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958302 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958307 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958311 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958314 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958317 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958320 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958324 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958328 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958331 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958334 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958338 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958341 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958344 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958347 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958350 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958354 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958357 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:17:20.981447 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958359 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958362 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958365 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958368 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958373 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958376 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958379 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958382 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958384 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958388 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958390 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958393 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958396 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958399 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958402 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958405 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958408 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958421 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958424 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:17:20.982133 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958427 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958430 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958432 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958436 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958439 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958441 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958444 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958447 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958450 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958453 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958457 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958460 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958466 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958469 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958472 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958475 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958477 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958482 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958485 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958488 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:17:21.048480 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958491 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:17:21.041558 ip-10-0-132-151 systemd[1]: Started Kubernetes Kubelet. Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958493 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958496 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958499 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958502 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958505 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958507 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958512 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958515 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958517 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958521 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958524 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958526 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958529 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958532 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958535 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958537 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958540 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958543 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958546 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:17:21.049893 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958548 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958553 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958556 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958559 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958561 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958564 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958567 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958570 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958573 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958577 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.958580 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.959441 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.970242 2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.970260 2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.970281 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 18:17:21.051915 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.970287 2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970355 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970363 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970367 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970371 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970374 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970378 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970381 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970384 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970387 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970390 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970393 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970396 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970400 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970403 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970406 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970409 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970412 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970415 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:17:21.052621 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970417 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970420 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970423 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970427 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970431 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970434 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970436 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970439 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970442 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970445 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970448 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970451 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970454 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970457 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970460 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970463 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970465 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970468 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970471 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970474 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:17:21.054562 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970476 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970479 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970482 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970485 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970488 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970491 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970494 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970496 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970499 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970502 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970505 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970508 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970511 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970514 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970517 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970520 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970523 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970526 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970529 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970531 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:17:21.055694 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970534 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970536 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970539 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970542 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970545 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970547 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970550 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970553 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970556 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970560 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970564 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970566 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970570 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970573 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970576 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970579 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970582 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970584 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970587 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970590 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:17:21.057138 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970593 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970596 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970600 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970604 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970608 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970611 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970614 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970617 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970620 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970631 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970634 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970637 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970640 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970643 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970645 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970648 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970651 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970653 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970656 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970658 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:17:21.058532 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.970661 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.970667 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.971094 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.971101 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.971104 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.971336 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.971341 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.971571 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.971576 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.971579 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.971801 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.971804 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.971808 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.971811 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.971814 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:17:21.059471 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972034 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972036 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972039 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972042 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972045 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972281 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972285 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972513 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972517 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972519 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972522 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972525 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972534 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972537 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972540 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972543 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972546 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972548 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972551 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:17:21.061763 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972556 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972559 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972563 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972566 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972569 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972573 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972576 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972579 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972582 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972585 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972589 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972593 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972596 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972599 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972602 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972605 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972607 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972610 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972613 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972616 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:17:21.063013 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972619 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972622 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972625 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972629 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972632 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972636 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972639 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972641 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972644 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972647 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972650 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972653 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972656 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972658 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972662 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972666 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972669 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972672 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972675 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:17:21.063991 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972678 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972681 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972684 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972686 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972690 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972693 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972695 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972698 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972701 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972704 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972706 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972709 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972711 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972714 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972717 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972720 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972723 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972726 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972730 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972733 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:17:21.064970 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972736 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:17:21.066762 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972739 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:17:21.066762 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972741 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:17:21.066762 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972744 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:17:21.066762 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972747 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:17:21.066762 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972750 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:17:21.066762 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972753 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:17:21.066762 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:20.972756 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:17:21.066762 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.972762 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:17:21.066762 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.972907 2574 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 18:17:21.066762 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.975299 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 18:17:21.066762 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.976777 2574 server.go:1019] "Starting client certificate rotation" Jul 10 18:17:21.066762 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.979949 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 18:17:21.066762 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.983317 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 18:17:21.067507 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.984367 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 18:17:21.067507 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.988197 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 18:17:21.067507 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:20.994043 2574 log.go:25] "Validated CRI v1 runtime API" Jul 10 18:17:21.067507 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.011912 2574 log.go:25] "Validated CRI v1 image API" Jul 10 18:17:21.067507 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.013530 2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 18:17:21.067507 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.016388 2574 fs.go:135] Filesystem UUIDs: map[2dc2b989-8d77-40a1-9ae8-a06419e770d1:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9665083b-80cb-4b78-8785-e8caefa9cffd:/dev/nvme0n1p3] Jul 10 18:17:21.067507 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.016405 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.023312 2574 manager.go:216] Machine: {Timestamp:2026-07-10 18:17:21.02180984 +0000 UTC m=+0.435094621 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098261 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec234477aa032e62856a69769a1f04e3 SystemUUID:ec234477-aa03-2e62-856a-69769a1f04e3 BootID:06924958-9f0e-4306-9595-a208eeae8351 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3e:f6:83:45:dd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3e:f6:83:45:dd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:86:1c:f2:56:55:a3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.023432 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.023582 2574 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.025038 2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.025075 2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-151.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.026004 2574 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.026020 2574 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.026032 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.026053 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.026976 2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.030018 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.030348 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.030527 2574 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.030563 2574 kubelet.go:486] "Attempting to sync node with API server" Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.030578 2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.030606 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 18:17:21.068978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.030621 2574 kubelet.go:398] "Adding apiserver pod source" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.030640 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.031079 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.031100 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.033630 2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.035289 2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.035322 2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.035936 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.035949 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.035955 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.035961 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.035970 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.035975 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.036151 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.036157 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.036164 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.036184 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.036194 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.036202 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.036968 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.036977 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.040523 2574 server.go:1268] "Started kubelet" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.040650 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.040721 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.040769 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.042443 2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.042884 2574 server.go:310] "Adding debug handlers to kubelet server" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.045558 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.045581 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-151.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.045727 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-151.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.045867 2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.049014 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.049033 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.049299 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.053071 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.053150 2574 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.053317 2574 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.048721 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-151.ec2.internal.18c10077966f9ea9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-151.ec2.internal,UID:ip-10-0-132-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-151.ec2.internal,},FirstTimestamp:2026-07-10 18:17:21.040432809 +0000 UTC m=+0.453717590,LastTimestamp:2026-07-10 18:17:21.040432809 +0000 UTC m=+0.453717590,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-151.ec2.internal,}" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.053503 2574 reconstruct.go:97] "Volume reconstruction finished" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.053510 2574 reconciler.go:29] "Reconciler: start to sync state" Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.053540 2574 factory.go:55] Registering systemd factory Jul 10 18:17:21.070419 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.053557 2574 factory.go:223] Registration of the systemd container factory successfully Jul 10 18:17:21.072159 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.055050 2574 factory.go:153] Registering CRI-O factory Jul 10 18:17:21.072159 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.055068 2574 factory.go:223] Registration of the crio container factory successfully Jul 10 18:17:21.072159 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.055140 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 18:17:21.072159 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.055193 2574 factory.go:103] Registering Raw factory Jul 10 18:17:21.072159 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.055211 2574 manager.go:1196] Started watching for new ooms in manager Jul 10 18:17:21.072159 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.055804 2574 manager.go:349] Starting recovery of all containers Jul 10 18:17:21.072159 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.061291 2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-151.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 18:17:21.072159 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.061373 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 18:17:21.072159 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.068087 2574 manager.go:354] Recovery completed Jul 10 18:17:21.076067 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.074570 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:17:21.077904 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.076655 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:17:21.077904 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.076690 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:17:21.077904 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.076705 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:17:21.077904 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.077371 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 18:17:21.077904 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.077383 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 18:17:21.077904 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.077389 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:17:21.077904 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.077399 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 10 18:17:21.157930 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.078767 2574 policy_none.go:49] "None policy: Start" Jul 10 18:17:21.157930 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.078781 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 18:17:21.157930 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.078794 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 18:17:21.157930 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.079496 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-151.ec2.internal.18c10077989892aa default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-151.ec2.internal,UID:ip-10-0-132-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-151.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-151.ec2.internal,},FirstTimestamp:2026-07-10 18:17:21.076671146 +0000 UTC m=+0.489955930,LastTimestamp:2026-07-10 18:17:21.076671146 +0000 UTC m=+0.489955930,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-151.ec2.internal,}" Jul 10 18:17:21.157930 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.079952 2574 policy_none.go:47] "Start" Jul 10 18:17:21.157930 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.081663 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:17:21.157930 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.081702 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:17:21.157930 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.081717 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:17:21.157930 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.100263 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-151.ec2.internal.18c100779898f962 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-151.ec2.internal,UID:ip-10-0-132-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-151.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-151.ec2.internal,},FirstTimestamp:2026-07-10 18:17:21.076697442 +0000 UTC m=+0.489982223,LastTimestamp:2026-07-10 18:17:21.076697442 +0000 UTC m=+0.489982223,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-151.ec2.internal,}" Jul 10 18:17:21.157930 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.120545 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-151.ec2.internal.18c100779899286e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-151.ec2.internal,UID:ip-10-0-132-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-151.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-151.ec2.internal,},FirstTimestamp:2026-07-10 18:17:21.076709486 +0000 UTC m=+0.489994267,LastTimestamp:2026-07-10 18:17:21.076709486 +0000 UTC m=+0.489994267,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-151.ec2.internal,}" Jul 10 18:17:21.157930 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.132817 2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-151.ec2.internal.18c10077989892aa\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-151.ec2.internal.18c10077989892aa default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-151.ec2.internal,UID:ip-10-0-132-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-151.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-151.ec2.internal,},FirstTimestamp:2026-07-10 18:17:21.076671146 +0000 UTC m=+0.489955930,LastTimestamp:2026-07-10 18:17:21.081683823 +0000 UTC m=+0.494968606,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-151.ec2.internal,}" Jul 10 18:17:21.157930 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.133530 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jz7wx" Jul 10 18:17:21.157930 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.155251 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 10 18:17:21.166492 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.166198 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jz7wx" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.169377 2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-151.ec2.internal.18c100779898f962\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-151.ec2.internal.18c100779898f962 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-151.ec2.internal,UID:ip-10-0-132-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-151.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-151.ec2.internal,},FirstTimestamp:2026-07-10 18:17:21.076697442 +0000 UTC m=+0.489982223,LastTimestamp:2026-07-10 18:17:21.081708779 +0000 UTC m=+0.494993562,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-151.ec2.internal,}" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.170039 2574 manager.go:337] "Starting Device Plugin manager" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.170082 2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.170112 2574 server.go:87] "Starting device plugin registration server" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.170407 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.170420 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.170533 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.170617 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.170628 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.171251 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.171298 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-151.ec2.internal\" not found" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.269356 2574 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-151.ec2.internal\" not found" node="ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.271486 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.272554 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.272585 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.272603 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:17:21.284081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.272631 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.399421 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.287905 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.399421 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.287934 2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-151.ec2.internal\": node \"ip-10-0-132-151.ec2.internal\" not found" Jul 10 18:17:21.399421 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.307422 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 18:17:21.399421 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.308654 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 18:17:21.399421 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.308691 2574 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 18:17:21.399421 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.308713 2574 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 18:17:21.399421 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.308747 2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 18:17:21.399421 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.311774 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:17:21.399421 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.316111 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.413151 2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal"] Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.414133 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.416088 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.416136 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.416150 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.417424 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.417860 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.417885 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.418611 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.418637 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.418647 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.418848 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.418861 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.418870 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.420052 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.420074 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.420497 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.420944 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.420970 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.420986 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.457452 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/17653f02796255e2c412a08e496ac872-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"17653f02796255e2c412a08e496ac872\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.457480 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17653f02796255e2c412a08e496ac872-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"17653f02796255e2c412a08e496ac872\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.457498 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7554e05d30e68919ee44413a4c1cb32c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-151.ec2.internal\" (UID: \"7554e05d30e68919ee44413a4c1cb32c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.462280 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-151.ec2.internal\" not found" node="ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.515302 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.472998 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-151.ec2.internal\" not found" node="ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.521529 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.520964 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 10 18:17:21.667804 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.557720 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7554e05d30e68919ee44413a4c1cb32c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-151.ec2.internal\" (UID: \"7554e05d30e68919ee44413a4c1cb32c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.667804 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.557752 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/17653f02796255e2c412a08e496ac872-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"17653f02796255e2c412a08e496ac872\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.667804 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.557771 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17653f02796255e2c412a08e496ac872-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"17653f02796255e2c412a08e496ac872\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.667804 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.557814 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17653f02796255e2c412a08e496ac872-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"17653f02796255e2c412a08e496ac872\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.667804 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.557857 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/17653f02796255e2c412a08e496ac872-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"17653f02796255e2c412a08e496ac872\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.667804 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.557843 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7554e05d30e68919ee44413a4c1cb32c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-151.ec2.internal\" (UID: \"7554e05d30e68919ee44413a4c1cb32c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.667804 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.623191 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 10 18:17:21.791174 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.726715 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 10 18:17:21.791174 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.764033 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.791174 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.775850 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 10 18:17:21.902907 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.827503 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 10 18:17:21.926214 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.926189 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:17:21.928565 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:21.928547 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 10 18:17:21.964317 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.964296 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:17:21.983667 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.983638 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 18:17:21.983871 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.983845 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 18:17:21.983920 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.983843 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 18:17:21.983920 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:21.983843 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 18:17:22.031416 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.031392 2574 apiserver.go:52] "Watching apiserver" Jul 10 18:17:22.049339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.049266 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 10 18:17:22.049339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.049267 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 18:17:22.055087 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.055059 2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 18:17:22.055449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.055429 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-nj84h","openshift-multus/multus-r98mv","openshift-multus/network-metrics-daemon-j7bj7","openshift-network-operator/iptables-alerter-wrf88","kube-system/global-pull-secret-syncer-84n7b","openshift-dns/node-resolver-7qhmx","openshift-network-diagnostics/network-check-target-56zsp","openshift-ovn-kubernetes/ovnkube-node-9c9bx","kube-system/konnectivity-agent-9rhl2","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2","openshift-cluster-node-tuning-operator/tuned-rl7np","openshift-image-registry/node-ca-c8b4p"] Jul 10 18:17:22.057196 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.057178 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.058745 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.058723 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.059499 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.059482 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 18:17:22.059668 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.059653 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 18:17:22.059710 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.059679 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 18:17:22.059742 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.059711 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 18:17:22.059787 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.059760 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 18:17:22.060266 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.060248 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g5z8r\"" Jul 10 18:17:22.060407 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.060393 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:22.060481 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.060463 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:22.061095 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.061080 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 18:17:22.061152 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.061122 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-t95gg\"" Jul 10 18:17:22.061757 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.061742 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wrf88" Jul 10 18:17:22.063055 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.063039 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:22.063130 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.063086 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:22.063903 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.063890 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 18:17:22.063973 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.063951 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 10 18:17:22.064141 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.064124 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:17:22.064582 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.064563 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7qhmx" Jul 10 18:17:22.064819 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.064795 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v2zrk\"" Jul 10 18:17:22.065008 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.064993 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 18:17:22.065153 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.065134 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 18:17:22.066133 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.066114 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:22.066221 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.066203 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:22.067251 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.066850 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 18:17:22.067251 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.066890 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 18:17:22.068913 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.067760 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-49tkz\"" Jul 10 18:17:22.068913 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.067990 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 18:17:22.069863 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.069825 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.071692 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.071675 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9rhl2" Jul 10 18:17:22.073142 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.073124 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.073455 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.073430 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 18:17:22.073455 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.073454 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k2b72\"" Jul 10 18:17:22.073610 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.073468 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 18:17:22.073610 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.073543 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 18:17:22.073610 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.073594 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 18:17:22.073750 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.073685 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 18:17:22.073839 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.073816 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 18:17:22.074374 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.074357 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 18:17:22.074374 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.074362 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lw2wh\"" Jul 10 18:17:22.074524 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.074405 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 18:17:22.074675 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.074639 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.075467 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.075449 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 18:17:22.075548 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.075489 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 18:17:22.075548 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.075517 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 18:17:22.075966 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.075952 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal"] Jul 10 18:17:22.076033 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.076023 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c8b4p" Jul 10 18:17:22.078000 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.077985 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal"] Jul 10 18:17:22.080030 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.080008 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 18:17:22.080162 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.080080 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:17:22.080162 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.080131 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 18:17:22.080273 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.080162 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 18:17:22.080273 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.080203 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:17:22.080415 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.080401 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 18:17:22.080469 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.080429 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 18:17:22.080469 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.080439 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-z7qt9\"" Jul 10 18:17:22.080469 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.080462 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ps8n2\"" Jul 10 18:17:22.080616 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.080515 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cp2qn\"" Jul 10 18:17:22.095581 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.095561 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rftp7" Jul 10 18:17:22.107770 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.107754 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rftp7" Jul 10 18:17:22.156913 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.156884 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 18:17:22.159971 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.159946 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsjf5\" (UniqueName: \"kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5\") pod \"network-check-target-56zsp\" (UID: \"c453daa4-8e53-47b8-a056-7a7c32164c45\") " pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:22.160111 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.159984 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-socket-dir\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.160111 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160009 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-sysconfig\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.160111 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160032 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-slash\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.160111 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160055 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0d031c2a-1702-4f4e-a90a-6e6767ebe663-ovnkube-config\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.160111 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160076 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-system-cni-dir\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.160111 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160097 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-cni-binary-copy\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.160441 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160139 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a73fabd7-58c7-4f58-883c-1fd0a6650512-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.160441 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160166 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a315c03f-2c33-4fc6-8cb8-419a77746c77-iptables-alerter-script\") pod \"iptables-alerter-wrf88\" (UID: \"a315c03f-2c33-4fc6-8cb8-419a77746c77\") " pod="openshift-network-operator/iptables-alerter-wrf88" Jul 10 18:17:22.160441 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160192 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-var-lib-openvswitch\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.160441 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160216 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-node-log\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.160441 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160249 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-run-ovn-kubernetes\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.160441 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160273 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-var-lib-kubelet\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.160441 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160298 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-hostroot\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.160441 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160331 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/42571c86-4eaf-44a2-868e-1e023a349549-hosts-file\") pod \"node-resolver-7qhmx\" (UID: \"42571c86-4eaf-44a2-868e-1e023a349549\") " pod="openshift-dns/node-resolver-7qhmx" Jul 10 18:17:22.160441 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160355 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-sysctl-d\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.160441 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160380 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a73fabd7-58c7-4f58-883c-1fd0a6650512-os-release\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.160441 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160428 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/97b02d51-cfd3-4d11-a108-c54bf9dca12e-dbus\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160457 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160485 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thmsm\" (UniqueName: \"kubernetes.io/projected/0d031c2a-1702-4f4e-a90a-6e6767ebe663-kube-api-access-thmsm\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160511 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pqdf\" (UniqueName: \"kubernetes.io/projected/1e02735c-c7f7-4070-9a1f-0410bace9cd9-kube-api-access-9pqdf\") pod \"node-ca-c8b4p\" (UID: \"1e02735c-c7f7-4070-9a1f-0410bace9cd9\") " pod="openshift-image-registry/node-ca-c8b4p" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160545 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-sysctl-conf\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160571 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-etc-openvswitch\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160600 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-log-socket\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160633 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-multus-conf-dir\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160665 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-run-k8s-cni-cncf-io\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160685 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-run-netns\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160704 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-etc-selinux\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160745 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-var-lib-kubelet\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160789 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-cnibin\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160820 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-run-multus-certs\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160846 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72gw8\" (UniqueName: \"kubernetes.io/projected/42571c86-4eaf-44a2-868e-1e023a349549-kube-api-access-72gw8\") pod \"node-resolver-7qhmx\" (UID: \"42571c86-4eaf-44a2-868e-1e023a349549\") " pod="openshift-dns/node-resolver-7qhmx" Jul 10 18:17:22.160870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160870 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-lib-modules\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160909 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-tuned\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160934 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42571c86-4eaf-44a2-868e-1e023a349549-tmp-dir\") pod \"node-resolver-7qhmx\" (UID: \"42571c86-4eaf-44a2-868e-1e023a349549\") " pod="openshift-dns/node-resolver-7qhmx" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160951 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/952ce2e7-c21f-4fe0-9357-ced0cedef546-konnectivity-ca\") pod \"konnectivity-agent-9rhl2\" (UID: \"952ce2e7-c21f-4fe0-9357-ced0cedef546\") " pod="kube-system/konnectivity-agent-9rhl2" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160976 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-modprobe-d\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.160998 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a73fabd7-58c7-4f58-883c-1fd0a6650512-cnibin\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161022 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r56z\" (UniqueName: \"kubernetes.io/projected/a315c03f-2c33-4fc6-8cb8-419a77746c77-kube-api-access-2r56z\") pod \"iptables-alerter-wrf88\" (UID: \"a315c03f-2c33-4fc6-8cb8-419a77746c77\") " pod="openshift-network-operator/iptables-alerter-wrf88" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161037 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-run-ovn\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161058 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-cni-bin\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161076 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0d031c2a-1702-4f4e-a90a-6e6767ebe663-env-overrides\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161090 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/952ce2e7-c21f-4fe0-9357-ced0cedef546-agent-certs\") pod \"konnectivity-agent-9rhl2\" (UID: \"952ce2e7-c21f-4fe0-9357-ced0cedef546\") " pod="kube-system/konnectivity-agent-9rhl2" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161126 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-sys\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161144 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/97b02d51-cfd3-4d11-a108-c54bf9dca12e-kubelet-config\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161166 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-run-openvswitch\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161184 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q59x7\" (UniqueName: \"kubernetes.io/projected/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-kube-api-access-q59x7\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161214 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-device-dir\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.161330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161260 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-run\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161300 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgpwg\" (UniqueName: \"kubernetes.io/projected/a73fabd7-58c7-4f58-883c-1fd0a6650512-kube-api-access-fgpwg\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161331 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-kubelet\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161349 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-run-netns\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161405 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-cni-netd\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161429 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0d031c2a-1702-4f4e-a90a-6e6767ebe663-ovnkube-script-lib\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161454 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-systemd\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161484 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a73fabd7-58c7-4f58-883c-1fd0a6650512-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161521 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0d031c2a-1702-4f4e-a90a-6e6767ebe663-ovn-node-metrics-cert\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161556 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-multus-cni-dir\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161587 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-tmp\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161608 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a315c03f-2c33-4fc6-8cb8-419a77746c77-host-slash\") pod \"iptables-alerter-wrf88\" (UID: \"a315c03f-2c33-4fc6-8cb8-419a77746c77\") " pod="openshift-network-operator/iptables-alerter-wrf88" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161623 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161638 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-run-systemd\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161651 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e02735c-c7f7-4070-9a1f-0410bace9cd9-host\") pod \"node-ca-c8b4p\" (UID: \"1e02735c-c7f7-4070-9a1f-0410bace9cd9\") " pod="openshift-image-registry/node-ca-c8b4p" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161678 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-os-release\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.161870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161700 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-multus-socket-dir-parent\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161716 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs\") pod \"network-metrics-daemon-j7bj7\" (UID: \"cfcce616-ce21-4462-865f-116401b2c78f\") " pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161740 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161764 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-host\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161789 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a73fabd7-58c7-4f58-883c-1fd0a6650512-system-cni-dir\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161815 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a73fabd7-58c7-4f58-883c-1fd0a6650512-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161840 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-multus-daemon-config\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161862 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-kubernetes\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161883 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a73fabd7-58c7-4f58-883c-1fd0a6650512-cni-binary-copy\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161906 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-systemd-units\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161929 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-sys-fs\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161951 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvqjc\" (UniqueName: \"kubernetes.io/projected/e9672184-d9a7-4913-9481-a10bbd1e7297-kube-api-access-dvqjc\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.161975 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e02735c-c7f7-4070-9a1f-0410bace9cd9-serviceca\") pod \"node-ca-c8b4p\" (UID: \"1e02735c-c7f7-4070-9a1f-0410bace9cd9\") " pod="openshift-image-registry/node-ca-c8b4p" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.162005 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-var-lib-cni-bin\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.162031 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-registration-dir\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.162056 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g828z\" (UniqueName: \"kubernetes.io/projected/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-kube-api-access-g828z\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.162456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.162075 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-var-lib-cni-multus\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.162982 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.162094 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-etc-kubernetes\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.162982 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.162128 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrzx4\" (UniqueName: \"kubernetes.io/projected/cfcce616-ce21-4462-865f-116401b2c78f-kube-api-access-zrzx4\") pod \"network-metrics-daemon-j7bj7\" (UID: \"cfcce616-ce21-4462-865f-116401b2c78f\") " pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:22.171643 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.171579 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 18:12:21 +0000 UTC" deadline="2028-02-08 15:07:24.280234672 +0000 UTC" Jul 10 18:17:22.171643 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.171638 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13868h50m2.108599565s" Jul 10 18:17:22.262401 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262353 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q59x7\" (UniqueName: \"kubernetes.io/projected/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-kube-api-access-q59x7\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.262401 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262409 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-device-dir\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.262829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262470 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-device-dir\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.262829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262545 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-run\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.262829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262574 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgpwg\" (UniqueName: \"kubernetes.io/projected/a73fabd7-58c7-4f58-883c-1fd0a6650512-kube-api-access-fgpwg\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.262829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262673 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-kubelet\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.262829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262699 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-run-netns\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.262829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262720 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-cni-netd\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.262829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262755 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0d031c2a-1702-4f4e-a90a-6e6767ebe663-ovnkube-script-lib\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.262829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262757 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-run\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.262829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262781 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-systemd\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.262829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262802 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-run-netns\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.262829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262760 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-kubelet\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.262829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262821 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-cni-netd\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.262829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262833 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a73fabd7-58c7-4f58-883c-1fd0a6650512-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262859 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0d031c2a-1702-4f4e-a90a-6e6767ebe663-ovn-node-metrics-cert\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262883 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-systemd\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262885 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-multus-cni-dir\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262922 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-tmp\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262945 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a315c03f-2c33-4fc6-8cb8-419a77746c77-host-slash\") pod \"iptables-alerter-wrf88\" (UID: \"a315c03f-2c33-4fc6-8cb8-419a77746c77\") " pod="openshift-network-operator/iptables-alerter-wrf88" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.262986 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263010 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-run-systemd\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263035 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e02735c-c7f7-4070-9a1f-0410bace9cd9-host\") pod \"node-ca-c8b4p\" (UID: \"1e02735c-c7f7-4070-9a1f-0410bace9cd9\") " pod="openshift-image-registry/node-ca-c8b4p" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263027 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-multus-cni-dir\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263060 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-os-release\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263085 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-multus-socket-dir-parent\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263099 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-run-systemd\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263111 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs\") pod \"network-metrics-daemon-j7bj7\" (UID: \"cfcce616-ce21-4462-865f-116401b2c78f\") " pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.263144 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263195 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-multus-socket-dir-parent\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.263252 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret podName:97b02d51-cfd3-4d11-a108-c54bf9dca12e nodeName:}" failed. No retries permitted until 2026-07-10 18:17:22.763207857 +0000 UTC m=+2.176492625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret") pod "global-pull-secret-syncer-84n7b" (UID: "97b02d51-cfd3-4d11-a108-c54bf9dca12e") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.263280 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:22.263630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263327 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.263342 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs podName:cfcce616-ce21-4462-865f-116401b2c78f nodeName:}" failed. No retries permitted until 2026-07-10 18:17:22.763325414 +0000 UTC m=+2.176610200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs") pod "network-metrics-daemon-j7bj7" (UID: "cfcce616-ce21-4462-865f-116401b2c78f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263278 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263354 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263372 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a315c03f-2c33-4fc6-8cb8-419a77746c77-host-slash\") pod \"iptables-alerter-wrf88\" (UID: \"a315c03f-2c33-4fc6-8cb8-419a77746c77\") " pod="openshift-network-operator/iptables-alerter-wrf88" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263374 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-host\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263405 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-os-release\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263410 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a73fabd7-58c7-4f58-883c-1fd0a6650512-system-cni-dir\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263414 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-host\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263448 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a73fabd7-58c7-4f58-883c-1fd0a6650512-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263455 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a73fabd7-58c7-4f58-883c-1fd0a6650512-system-cni-dir\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263346 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e02735c-c7f7-4070-9a1f-0410bace9cd9-host\") pod \"node-ca-c8b4p\" (UID: \"1e02735c-c7f7-4070-9a1f-0410bace9cd9\") " pod="openshift-image-registry/node-ca-c8b4p" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263477 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-multus-daemon-config\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263503 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-kubernetes\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263532 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a73fabd7-58c7-4f58-883c-1fd0a6650512-cni-binary-copy\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263561 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0d031c2a-1702-4f4e-a90a-6e6767ebe663-ovnkube-script-lib\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263476 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a73fabd7-58c7-4f58-883c-1fd0a6650512-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.264483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263559 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-systemd-units\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263629 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-systemd-units\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263637 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-kubernetes\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263639 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-sys-fs\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263680 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvqjc\" (UniqueName: \"kubernetes.io/projected/e9672184-d9a7-4913-9481-a10bbd1e7297-kube-api-access-dvqjc\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263703 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-sys-fs\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263706 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e02735c-c7f7-4070-9a1f-0410bace9cd9-serviceca\") pod \"node-ca-c8b4p\" (UID: \"1e02735c-c7f7-4070-9a1f-0410bace9cd9\") " pod="openshift-image-registry/node-ca-c8b4p" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263740 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-var-lib-cni-bin\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263788 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-registration-dir\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263827 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g828z\" (UniqueName: \"kubernetes.io/projected/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-kube-api-access-g828z\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263855 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-var-lib-cni-multus\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263898 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-etc-kubernetes\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263924 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrzx4\" (UniqueName: \"kubernetes.io/projected/cfcce616-ce21-4462-865f-116401b2c78f-kube-api-access-zrzx4\") pod \"network-metrics-daemon-j7bj7\" (UID: \"cfcce616-ce21-4462-865f-116401b2c78f\") " pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263952 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsjf5\" (UniqueName: \"kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5\") pod \"network-check-target-56zsp\" (UID: \"c453daa4-8e53-47b8-a056-7a7c32164c45\") " pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.263976 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-socket-dir\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264002 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-sysconfig\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264020 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a73fabd7-58c7-4f58-883c-1fd0a6650512-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.265326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264027 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-slash\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264061 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-slash\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264079 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0d031c2a-1702-4f4e-a90a-6e6767ebe663-ovnkube-config\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264103 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-var-lib-cni-bin\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264112 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-system-cni-dir\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264135 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e02735c-c7f7-4070-9a1f-0410bace9cd9-serviceca\") pod \"node-ca-c8b4p\" (UID: \"1e02735c-c7f7-4070-9a1f-0410bace9cd9\") " pod="openshift-image-registry/node-ca-c8b4p" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264141 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-cni-binary-copy\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264161 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-var-lib-cni-multus\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264175 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a73fabd7-58c7-4f58-883c-1fd0a6650512-cni-binary-copy\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264167 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a73fabd7-58c7-4f58-883c-1fd0a6650512-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264106 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-etc-kubernetes\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264244 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-sysconfig\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264313 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a73fabd7-58c7-4f58-883c-1fd0a6650512-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264060 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-multus-daemon-config\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264489 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-registration-dir\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264496 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-socket-dir\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264520 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a315c03f-2c33-4fc6-8cb8-419a77746c77-iptables-alerter-script\") pod \"iptables-alerter-wrf88\" (UID: \"a315c03f-2c33-4fc6-8cb8-419a77746c77\") " pod="openshift-network-operator/iptables-alerter-wrf88" Jul 10 18:17:22.266150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264544 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-system-cni-dir\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264585 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-var-lib-openvswitch\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264626 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-node-log\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264653 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-run-ovn-kubernetes\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264692 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-var-lib-kubelet\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264732 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-hostroot\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264759 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/42571c86-4eaf-44a2-868e-1e023a349549-hosts-file\") pod \"node-resolver-7qhmx\" (UID: \"42571c86-4eaf-44a2-868e-1e023a349549\") " pod="openshift-dns/node-resolver-7qhmx" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264783 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-sysctl-d\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264825 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a73fabd7-58c7-4f58-883c-1fd0a6650512-os-release\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264853 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/97b02d51-cfd3-4d11-a108-c54bf9dca12e-dbus\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264880 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264909 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thmsm\" (UniqueName: \"kubernetes.io/projected/0d031c2a-1702-4f4e-a90a-6e6767ebe663-kube-api-access-thmsm\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264935 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pqdf\" (UniqueName: \"kubernetes.io/projected/1e02735c-c7f7-4070-9a1f-0410bace9cd9-kube-api-access-9pqdf\") pod \"node-ca-c8b4p\" (UID: \"1e02735c-c7f7-4070-9a1f-0410bace9cd9\") " pod="openshift-image-registry/node-ca-c8b4p" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264962 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-sysctl-conf\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264975 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-cni-binary-copy\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.264989 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-etc-openvswitch\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265031 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/42571c86-4eaf-44a2-868e-1e023a349549-hosts-file\") pod \"node-resolver-7qhmx\" (UID: \"42571c86-4eaf-44a2-868e-1e023a349549\") " pod="openshift-dns/node-resolver-7qhmx" Jul 10 18:17:22.267339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265072 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-log-socket\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265095 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-multus-conf-dir\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265112 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a315c03f-2c33-4fc6-8cb8-419a77746c77-iptables-alerter-script\") pod \"iptables-alerter-wrf88\" (UID: \"a315c03f-2c33-4fc6-8cb8-419a77746c77\") " pod="openshift-network-operator/iptables-alerter-wrf88" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265123 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-run-k8s-cni-cncf-io\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265138 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-sysctl-conf\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265173 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-run-netns\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265187 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-etc-openvswitch\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265196 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-var-lib-kubelet\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265204 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-etc-selinux\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265241 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a73fabd7-58c7-4f58-883c-1fd0a6650512-os-release\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265246 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-var-lib-kubelet\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265274 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-cnibin\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265278 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e9672184-d9a7-4913-9481-a10bbd1e7297-etc-selinux\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265290 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/97b02d51-cfd3-4d11-a108-c54bf9dca12e-dbus\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265330 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-run-k8s-cni-cncf-io\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265335 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-run-multus-certs\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265365 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72gw8\" (UniqueName: \"kubernetes.io/projected/42571c86-4eaf-44a2-868e-1e023a349549-kube-api-access-72gw8\") pod \"node-resolver-7qhmx\" (UID: \"42571c86-4eaf-44a2-868e-1e023a349549\") " pod="openshift-dns/node-resolver-7qhmx" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265396 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-lib-modules\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.268007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265424 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-tuned\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265438 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-cnibin\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265453 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42571c86-4eaf-44a2-868e-1e023a349549-tmp-dir\") pod \"node-resolver-7qhmx\" (UID: \"42571c86-4eaf-44a2-868e-1e023a349549\") " pod="openshift-dns/node-resolver-7qhmx" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265475 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-var-lib-openvswitch\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265483 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/952ce2e7-c21f-4fe0-9357-ced0cedef546-konnectivity-ca\") pod \"konnectivity-agent-9rhl2\" (UID: \"952ce2e7-c21f-4fe0-9357-ced0cedef546\") " pod="kube-system/konnectivity-agent-9rhl2" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265366 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-run-netns\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265397 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-hostroot\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265531 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265638 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-sysctl-d\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265676 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-run-ovn-kubernetes\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265693 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-multus-conf-dir\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265716 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-var-lib-kubelet\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265737 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-node-log\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265799 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-modprobe-d\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265838 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a73fabd7-58c7-4f58-883c-1fd0a6650512-cnibin\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265867 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r56z\" (UniqueName: \"kubernetes.io/projected/a315c03f-2c33-4fc6-8cb8-419a77746c77-kube-api-access-2r56z\") pod \"iptables-alerter-wrf88\" (UID: \"a315c03f-2c33-4fc6-8cb8-419a77746c77\") " pod="openshift-network-operator/iptables-alerter-wrf88" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265928 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-run-ovn\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265956 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-cni-bin\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.268651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.265983 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0d031c2a-1702-4f4e-a90a-6e6767ebe663-env-overrides\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266010 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/952ce2e7-c21f-4fe0-9357-ced0cedef546-agent-certs\") pod \"konnectivity-agent-9rhl2\" (UID: \"952ce2e7-c21f-4fe0-9357-ced0cedef546\") " pod="kube-system/konnectivity-agent-9rhl2" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266037 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-sys\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266065 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/97b02d51-cfd3-4d11-a108-c54bf9dca12e-kubelet-config\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266092 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-run-openvswitch\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266224 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/952ce2e7-c21f-4fe0-9357-ced0cedef546-konnectivity-ca\") pod \"konnectivity-agent-9rhl2\" (UID: \"952ce2e7-c21f-4fe0-9357-ced0cedef546\") " pod="kube-system/konnectivity-agent-9rhl2" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266258 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-run-openvswitch\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266423 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-log-socket\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266508 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-modprobe-d\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266523 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0d031c2a-1702-4f4e-a90a-6e6767ebe663-ovnkube-config\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266560 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-lib-modules\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266563 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a73fabd7-58c7-4f58-883c-1fd0a6650512-cnibin\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266594 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-run-ovn\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266620 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-host-run-multus-certs\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266642 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-sys\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266642 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d031c2a-1702-4f4e-a90a-6e6767ebe663-host-cni-bin\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.266738 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/97b02d51-cfd3-4d11-a108-c54bf9dca12e-kubelet-config\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.267154 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42571c86-4eaf-44a2-868e-1e023a349549-tmp-dir\") pod \"node-resolver-7qhmx\" (UID: \"42571c86-4eaf-44a2-868e-1e023a349549\") " pod="openshift-dns/node-resolver-7qhmx" Jul 10 18:17:22.269268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.267260 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0d031c2a-1702-4f4e-a90a-6e6767ebe663-env-overrides\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.269848 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.267706 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0d031c2a-1702-4f4e-a90a-6e6767ebe663-ovn-node-metrics-cert\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.269848 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.268445 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-tmp\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.269848 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.268463 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-etc-tuned\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.269848 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.269057 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/952ce2e7-c21f-4fe0-9357-ced0cedef546-agent-certs\") pod \"konnectivity-agent-9rhl2\" (UID: \"952ce2e7-c21f-4fe0-9357-ced0cedef546\") " pod="kube-system/konnectivity-agent-9rhl2" Jul 10 18:17:22.278441 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.278416 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgpwg\" (UniqueName: \"kubernetes.io/projected/a73fabd7-58c7-4f58-883c-1fd0a6650512-kube-api-access-fgpwg\") pod \"multus-additional-cni-plugins-nj84h\" (UID: \"a73fabd7-58c7-4f58-883c-1fd0a6650512\") " pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.282018 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.281990 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrzx4\" (UniqueName: \"kubernetes.io/projected/cfcce616-ce21-4462-865f-116401b2c78f-kube-api-access-zrzx4\") pod \"network-metrics-daemon-j7bj7\" (UID: \"cfcce616-ce21-4462-865f-116401b2c78f\") " pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:22.284461 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.284371 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g828z\" (UniqueName: \"kubernetes.io/projected/8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a-kube-api-access-g828z\") pod \"tuned-rl7np\" (UID: \"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a\") " pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.285420 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.284898 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvqjc\" (UniqueName: \"kubernetes.io/projected/e9672184-d9a7-4913-9481-a10bbd1e7297-kube-api-access-dvqjc\") pod \"aws-ebs-csi-driver-node-2wpd2\" (UID: \"e9672184-d9a7-4913-9481-a10bbd1e7297\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.285420 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.285084 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:17:22.285420 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.285103 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:17:22.285420 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.285115 2574 projected.go:196] Error preparing data for projected volume kube-api-access-tsjf5 for pod openshift-network-diagnostics/network-check-target-56zsp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:22.285420 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.285157 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r56z\" (UniqueName: \"kubernetes.io/projected/a315c03f-2c33-4fc6-8cb8-419a77746c77-kube-api-access-2r56z\") pod \"iptables-alerter-wrf88\" (UID: \"a315c03f-2c33-4fc6-8cb8-419a77746c77\") " pod="openshift-network-operator/iptables-alerter-wrf88" Jul 10 18:17:22.285420 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.285189 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thmsm\" (UniqueName: \"kubernetes.io/projected/0d031c2a-1702-4f4e-a90a-6e6767ebe663-kube-api-access-thmsm\") pod \"ovnkube-node-9c9bx\" (UID: \"0d031c2a-1702-4f4e-a90a-6e6767ebe663\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.285420 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.285200 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5 podName:c453daa4-8e53-47b8-a056-7a7c32164c45 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:22.785159255 +0000 UTC m=+2.198444022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tsjf5" (UniqueName: "kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5") pod "network-check-target-56zsp" (UID: "c453daa4-8e53-47b8-a056-7a7c32164c45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:22.285917 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.285683 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pqdf\" (UniqueName: \"kubernetes.io/projected/1e02735c-c7f7-4070-9a1f-0410bace9cd9-kube-api-access-9pqdf\") pod \"node-ca-c8b4p\" (UID: \"1e02735c-c7f7-4070-9a1f-0410bace9cd9\") " pod="openshift-image-registry/node-ca-c8b4p" Jul 10 18:17:22.285917 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.285700 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72gw8\" (UniqueName: \"kubernetes.io/projected/42571c86-4eaf-44a2-868e-1e023a349549-kube-api-access-72gw8\") pod \"node-resolver-7qhmx\" (UID: \"42571c86-4eaf-44a2-868e-1e023a349549\") " pod="openshift-dns/node-resolver-7qhmx" Jul 10 18:17:22.287691 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.287647 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q59x7\" (UniqueName: \"kubernetes.io/projected/c2a28dd9-4966-4c3d-8c68-bfacfe569bea-kube-api-access-q59x7\") pod \"multus-r98mv\" (UID: \"c2a28dd9-4966-4c3d-8c68-bfacfe569bea\") " pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.393748 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.393676 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nj84h" Jul 10 18:17:22.412869 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.412838 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r98mv" Jul 10 18:17:22.417487 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:22.417407 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7554e05d30e68919ee44413a4c1cb32c.slice/crio-762c84b7497b7d5195badda7090e7a4048e678c0c489463846a38b01e1a25787 WatchSource:0}: Error finding container 762c84b7497b7d5195badda7090e7a4048e678c0c489463846a38b01e1a25787: Status 404 returned error can't find the container with id 762c84b7497b7d5195badda7090e7a4048e678c0c489463846a38b01e1a25787 Jul 10 18:17:22.418265 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:22.418244 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda73fabd7_58c7_4f58_883c_1fd0a6650512.slice/crio-72c088d7ed8292e329ecd1ced0b86f54db192afa458feb6db22400e41bac90bb WatchSource:0}: Error finding container 72c088d7ed8292e329ecd1ced0b86f54db192afa458feb6db22400e41bac90bb: Status 404 returned error can't find the container with id 72c088d7ed8292e329ecd1ced0b86f54db192afa458feb6db22400e41bac90bb Jul 10 18:17:22.419095 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:22.419076 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17653f02796255e2c412a08e496ac872.slice/crio-4c7a7c462795dcdd7d745a293032c77e5ddd1662e8f9220a74da6a596ff3df5f WatchSource:0}: Error finding container 4c7a7c462795dcdd7d745a293032c77e5ddd1662e8f9220a74da6a596ff3df5f: Status 404 returned error can't find the container with id 4c7a7c462795dcdd7d745a293032c77e5ddd1662e8f9220a74da6a596ff3df5f Jul 10 18:17:22.423568 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.423549 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:17:22.424709 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:22.424683 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2a28dd9_4966_4c3d_8c68_bfacfe569bea.slice/crio-879368563f26ba615144e15711ecf1952bdba40a9a4a63231d0fdfc9fe4443fc WatchSource:0}: Error finding container 879368563f26ba615144e15711ecf1952bdba40a9a4a63231d0fdfc9fe4443fc: Status 404 returned error can't find the container with id 879368563f26ba615144e15711ecf1952bdba40a9a4a63231d0fdfc9fe4443fc Jul 10 18:17:22.429233 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.429213 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wrf88" Jul 10 18:17:22.441104 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:22.441084 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda315c03f_2c33_4fc6_8cb8_419a77746c77.slice/crio-ea8f64f93ea616a707caf80caf64fe8c6dac7f8eeb3ecbb5cfca98e6913e4071 WatchSource:0}: Error finding container ea8f64f93ea616a707caf80caf64fe8c6dac7f8eeb3ecbb5cfca98e6913e4071: Status 404 returned error can't find the container with id ea8f64f93ea616a707caf80caf64fe8c6dac7f8eeb3ecbb5cfca98e6913e4071 Jul 10 18:17:22.447125 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.447108 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7qhmx" Jul 10 18:17:22.456364 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:22.456336 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42571c86_4eaf_44a2_868e_1e023a349549.slice/crio-179fdd27022f0c89e8e5a1983fc5baae24967c01abdcc0d741d2158fb7acdfbf WatchSource:0}: Error finding container 179fdd27022f0c89e8e5a1983fc5baae24967c01abdcc0d741d2158fb7acdfbf: Status 404 returned error can't find the container with id 179fdd27022f0c89e8e5a1983fc5baae24967c01abdcc0d741d2158fb7acdfbf Jul 10 18:17:22.463798 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.463780 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:22.471957 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:22.471934 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d031c2a_1702_4f4e_a90a_6e6767ebe663.slice/crio-4030f9546c88e80b5495a85b3305a335996bf2d9ef2224f5b7e91f8862fb753c WatchSource:0}: Error finding container 4030f9546c88e80b5495a85b3305a335996bf2d9ef2224f5b7e91f8862fb753c: Status 404 returned error can't find the container with id 4030f9546c88e80b5495a85b3305a335996bf2d9ef2224f5b7e91f8862fb753c Jul 10 18:17:22.479283 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.479263 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9rhl2" Jul 10 18:17:22.487721 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:22.487697 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod952ce2e7_c21f_4fe0_9357_ced0cedef546.slice/crio-ac841c9cf72b024bbe1e63f279b8d95d35d119e88d56445d5033318dfb9e64fa WatchSource:0}: Error finding container ac841c9cf72b024bbe1e63f279b8d95d35d119e88d56445d5033318dfb9e64fa: Status 404 returned error can't find the container with id ac841c9cf72b024bbe1e63f279b8d95d35d119e88d56445d5033318dfb9e64fa Jul 10 18:17:22.497298 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.497274 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" Jul 10 18:17:22.505743 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:22.505715 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9672184_d9a7_4913_9481_a10bbd1e7297.slice/crio-f8f0aa3c59756760dc6eea7dd044ceae659b619de6eaf09764ab716dfcdd21ed WatchSource:0}: Error finding container f8f0aa3c59756760dc6eea7dd044ceae659b619de6eaf09764ab716dfcdd21ed: Status 404 returned error can't find the container with id f8f0aa3c59756760dc6eea7dd044ceae659b619de6eaf09764ab716dfcdd21ed Jul 10 18:17:22.512054 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.512035 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rl7np" Jul 10 18:17:22.516951 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.516935 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c8b4p" Jul 10 18:17:22.522279 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:22.522258 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cd76465_cc20_4ce3_b7c1_06c5bb9ee68a.slice/crio-a269d4f7f15cfb061024af30e276eaa402ea25cfe93b8618d44a37086c6fc30f WatchSource:0}: Error finding container a269d4f7f15cfb061024af30e276eaa402ea25cfe93b8618d44a37086c6fc30f: Status 404 returned error can't find the container with id a269d4f7f15cfb061024af30e276eaa402ea25cfe93b8618d44a37086c6fc30f Jul 10 18:17:22.526470 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:22.526448 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e02735c_c7f7_4070_9a1f_0410bace9cd9.slice/crio-e8b82ccbd55c8c61bcb5a4ddc9031c870b33f7216fd4717546e6a6bb73b0ceca WatchSource:0}: Error finding container e8b82ccbd55c8c61bcb5a4ddc9031c870b33f7216fd4717546e6a6bb73b0ceca: Status 404 returned error can't find the container with id e8b82ccbd55c8c61bcb5a4ddc9031c870b33f7216fd4717546e6a6bb73b0ceca Jul 10 18:17:22.528353 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.528328 2574 kuberuntime_manager.go:1459] "Unhandled Error" err="container node-ca start failed in pod node-ca-c8b4p_openshift-image-registry(1e02735c-c7f7-4070-9a1f-0410bace9cd9): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 10 18:17:22.528443 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.528371 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"node-ca\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-image-registry/node-ca-c8b4p" podUID="1e02735c-c7f7-4070-9a1f-0410bace9cd9" Jul 10 18:17:22.769954 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.769866 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:22.769954 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.769903 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs\") pod \"network-metrics-daemon-j7bj7\" (UID: \"cfcce616-ce21-4462-865f-116401b2c78f\") " pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:22.770177 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.770018 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:22.770177 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.770021 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:22.770177 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.770073 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs podName:cfcce616-ce21-4462-865f-116401b2c78f nodeName:}" failed. No retries permitted until 2026-07-10 18:17:23.770059639 +0000 UTC m=+3.183344406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs") pod "network-metrics-daemon-j7bj7" (UID: "cfcce616-ce21-4462-865f-116401b2c78f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:22.770177 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.770088 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret podName:97b02d51-cfd3-4d11-a108-c54bf9dca12e nodeName:}" failed. No retries permitted until 2026-07-10 18:17:23.770082094 +0000 UTC m=+3.183366860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret") pod "global-pull-secret-syncer-84n7b" (UID: "97b02d51-cfd3-4d11-a108-c54bf9dca12e") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:22.870787 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:22.870724 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsjf5\" (UniqueName: \"kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5\") pod \"network-check-target-56zsp\" (UID: \"c453daa4-8e53-47b8-a056-7a7c32164c45\") " pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:22.870958 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.870936 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:17:22.871009 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.870961 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:17:22.871009 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.870974 2574 projected.go:196] Error preparing data for projected volume kube-api-access-tsjf5 for pod openshift-network-diagnostics/network-check-target-56zsp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:22.871128 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:22.871036 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5 podName:c453daa4-8e53-47b8-a056-7a7c32164c45 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:23.871018913 +0000 UTC m=+3.284303682 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tsjf5" (UniqueName: "kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5") pod "network-check-target-56zsp" (UID: "c453daa4-8e53-47b8-a056-7a7c32164c45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:23.071386 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.071306 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:17:23.109701 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.109613 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 18:12:22 +0000 UTC" deadline="2027-12-08 14:11:53.895279893 +0000 UTC" Jul 10 18:17:23.109701 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.109698 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12379h54m30.785586523s" Jul 10 18:17:23.312977 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.312228 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:23.312977 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:23.312367 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:23.312977 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.312764 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:23.312977 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:23.312848 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:23.322224 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.322127 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9rhl2" event={"ID":"952ce2e7-c21f-4fe0-9357-ced0cedef546","Type":"ContainerStarted","Data":"ac841c9cf72b024bbe1e63f279b8d95d35d119e88d56445d5033318dfb9e64fa"} Jul 10 18:17:23.331606 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.331510 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" event={"ID":"0d031c2a-1702-4f4e-a90a-6e6767ebe663","Type":"ContainerStarted","Data":"4030f9546c88e80b5495a85b3305a335996bf2d9ef2224f5b7e91f8862fb753c"} Jul 10 18:17:23.333766 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.333740 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r98mv" event={"ID":"c2a28dd9-4966-4c3d-8c68-bfacfe569bea","Type":"ContainerStarted","Data":"879368563f26ba615144e15711ecf1952bdba40a9a4a63231d0fdfc9fe4443fc"} Jul 10 18:17:23.336565 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.336512 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nj84h" event={"ID":"a73fabd7-58c7-4f58-883c-1fd0a6650512","Type":"ContainerStarted","Data":"72c088d7ed8292e329ecd1ced0b86f54db192afa458feb6db22400e41bac90bb"} Jul 10 18:17:23.343088 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.343063 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c8b4p" event={"ID":"1e02735c-c7f7-4070-9a1f-0410bace9cd9","Type":"ContainerStarted","Data":"e8b82ccbd55c8c61bcb5a4ddc9031c870b33f7216fd4717546e6a6bb73b0ceca"} Jul 10 18:17:23.349694 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:23.349630 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"node-ca\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dfd7fe2428b045559018b37d81a278873fc80eb81bd66999d7c242352ca2cbbf\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-image-registry/node-ca-c8b4p" podUID="1e02735c-c7f7-4070-9a1f-0410bace9cd9" Jul 10 18:17:23.360270 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.360239 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rl7np" event={"ID":"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a","Type":"ContainerStarted","Data":"a269d4f7f15cfb061024af30e276eaa402ea25cfe93b8618d44a37086c6fc30f"} Jul 10 18:17:23.375949 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.375710 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" event={"ID":"e9672184-d9a7-4913-9481-a10bbd1e7297","Type":"ContainerStarted","Data":"f8f0aa3c59756760dc6eea7dd044ceae659b619de6eaf09764ab716dfcdd21ed"} Jul 10 18:17:23.386250 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.386165 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7qhmx" event={"ID":"42571c86-4eaf-44a2-868e-1e023a349549","Type":"ContainerStarted","Data":"179fdd27022f0c89e8e5a1983fc5baae24967c01abdcc0d741d2158fb7acdfbf"} Jul 10 18:17:23.390159 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.390076 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wrf88" event={"ID":"a315c03f-2c33-4fc6-8cb8-419a77746c77","Type":"ContainerStarted","Data":"ea8f64f93ea616a707caf80caf64fe8c6dac7f8eeb3ecbb5cfca98e6913e4071"} Jul 10 18:17:23.392196 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.392132 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" event={"ID":"17653f02796255e2c412a08e496ac872","Type":"ContainerStarted","Data":"4c7a7c462795dcdd7d745a293032c77e5ddd1662e8f9220a74da6a596ff3df5f"} Jul 10 18:17:23.402831 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.402802 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" event={"ID":"7554e05d30e68919ee44413a4c1cb32c","Type":"ContainerStarted","Data":"762c84b7497b7d5195badda7090e7a4048e678c0c489463846a38b01e1a25787"} Jul 10 18:17:23.779137 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.779030 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:23.779137 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.779089 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs\") pod \"network-metrics-daemon-j7bj7\" (UID: \"cfcce616-ce21-4462-865f-116401b2c78f\") " pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:23.779356 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:23.779275 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:23.779356 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:23.779341 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs podName:cfcce616-ce21-4462-865f-116401b2c78f nodeName:}" failed. No retries permitted until 2026-07-10 18:17:25.779322841 +0000 UTC m=+5.192607620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs") pod "network-metrics-daemon-j7bj7" (UID: "cfcce616-ce21-4462-865f-116401b2c78f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:23.779800 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:23.779781 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:23.779874 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:23.779840 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret podName:97b02d51-cfd3-4d11-a108-c54bf9dca12e nodeName:}" failed. No retries permitted until 2026-07-10 18:17:25.779824397 +0000 UTC m=+5.193109163 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret") pod "global-pull-secret-syncer-84n7b" (UID: "97b02d51-cfd3-4d11-a108-c54bf9dca12e") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:23.880331 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:23.880284 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsjf5\" (UniqueName: \"kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5\") pod \"network-check-target-56zsp\" (UID: \"c453daa4-8e53-47b8-a056-7a7c32164c45\") " pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:23.880733 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:23.880706 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:17:23.880733 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:23.880730 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:17:23.880865 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:23.880743 2574 projected.go:196] Error preparing data for projected volume kube-api-access-tsjf5 for pod openshift-network-diagnostics/network-check-target-56zsp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:23.880865 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:23.880809 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5 podName:c453daa4-8e53-47b8-a056-7a7c32164c45 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:25.880790046 +0000 UTC m=+5.294074826 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tsjf5" (UniqueName: "kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5") pod "network-check-target-56zsp" (UID: "c453daa4-8e53-47b8-a056-7a7c32164c45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:24.110023 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:24.109899 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 18:12:22 +0000 UTC" deadline="2027-12-22 14:28:49.445807379 +0000 UTC" Jul 10 18:17:24.110023 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:24.109975 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12716h11m25.335836302s" Jul 10 18:17:24.309613 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:24.309582 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:24.309787 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:24.309711 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:24.596538 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:24.596461 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:17:24.926724 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:24.926647 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:17:25.316297 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:25.316198 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:25.316839 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:25.316208 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:25.316839 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:25.316320 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:25.316839 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:25.316431 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:25.797228 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:25.797139 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:25.797228 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:25.797190 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs\") pod \"network-metrics-daemon-j7bj7\" (UID: \"cfcce616-ce21-4462-865f-116401b2c78f\") " pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:25.797436 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:25.797364 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:25.797436 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:25.797425 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs podName:cfcce616-ce21-4462-865f-116401b2c78f nodeName:}" failed. No retries permitted until 2026-07-10 18:17:29.797404291 +0000 UTC m=+9.210689070 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs") pod "network-metrics-daemon-j7bj7" (UID: "cfcce616-ce21-4462-865f-116401b2c78f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:25.797815 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:25.797794 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:25.797898 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:25.797853 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret podName:97b02d51-cfd3-4d11-a108-c54bf9dca12e nodeName:}" failed. No retries permitted until 2026-07-10 18:17:29.797837726 +0000 UTC m=+9.211122496 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret") pod "global-pull-secret-syncer-84n7b" (UID: "97b02d51-cfd3-4d11-a108-c54bf9dca12e") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:25.897632 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:25.897596 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsjf5\" (UniqueName: \"kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5\") pod \"network-check-target-56zsp\" (UID: \"c453daa4-8e53-47b8-a056-7a7c32164c45\") " pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:25.897818 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:25.897778 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:17:25.897818 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:25.897801 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:17:25.897818 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:25.897814 2574 projected.go:196] Error preparing data for projected volume kube-api-access-tsjf5 for pod openshift-network-diagnostics/network-check-target-56zsp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:25.897981 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:25.897880 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5 podName:c453daa4-8e53-47b8-a056-7a7c32164c45 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:29.897859554 +0000 UTC m=+9.311144345 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tsjf5" (UniqueName: "kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5") pod "network-check-target-56zsp" (UID: "c453daa4-8e53-47b8-a056-7a7c32164c45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:26.309734 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:26.309703 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:26.309900 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:26.309867 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:27.313352 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:27.313283 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:27.313795 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:27.313452 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:27.313867 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:27.313849 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:27.313988 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:27.313965 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:28.309220 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:28.309177 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:28.309416 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:28.309357 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:29.312359 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:29.312328 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:29.312765 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:29.312328 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:29.312765 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:29.312478 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:29.312765 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:29.312552 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:29.829941 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:29.829304 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:29.829941 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:29.829359 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs\") pod \"network-metrics-daemon-j7bj7\" (UID: \"cfcce616-ce21-4462-865f-116401b2c78f\") " pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:29.829941 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:29.829531 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:29.829941 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:29.829559 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:29.829941 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:29.829596 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs podName:cfcce616-ce21-4462-865f-116401b2c78f nodeName:}" failed. No retries permitted until 2026-07-10 18:17:37.829576534 +0000 UTC m=+17.242861329 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs") pod "network-metrics-daemon-j7bj7" (UID: "cfcce616-ce21-4462-865f-116401b2c78f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:29.829941 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:29.829626 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret podName:97b02d51-cfd3-4d11-a108-c54bf9dca12e nodeName:}" failed. No retries permitted until 2026-07-10 18:17:37.829606906 +0000 UTC m=+17.242891673 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret") pod "global-pull-secret-syncer-84n7b" (UID: "97b02d51-cfd3-4d11-a108-c54bf9dca12e") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:29.930509 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:29.930458 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsjf5\" (UniqueName: \"kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5\") pod \"network-check-target-56zsp\" (UID: \"c453daa4-8e53-47b8-a056-7a7c32164c45\") " pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:29.930694 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:29.930672 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:17:29.930694 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:29.930692 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:17:29.930803 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:29.930706 2574 projected.go:196] Error preparing data for projected volume kube-api-access-tsjf5 for pod openshift-network-diagnostics/network-check-target-56zsp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:29.930803 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:29.930766 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5 podName:c453daa4-8e53-47b8-a056-7a7c32164c45 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:37.930747897 +0000 UTC m=+17.344032681 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tsjf5" (UniqueName: "kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5") pod "network-check-target-56zsp" (UID: "c453daa4-8e53-47b8-a056-7a7c32164c45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:30.309333 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:30.309252 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:30.309506 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:30.309369 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:31.310706 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:31.310145 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:31.310706 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:31.310254 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:31.310706 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:31.310555 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:31.310706 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:31.310657 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:32.309723 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:32.309681 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:32.309900 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:32.309821 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:33.309648 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:33.309615 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:33.310113 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:33.309620 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:33.310113 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:33.309758 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:33.310113 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:33.309811 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:34.309057 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:34.309019 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:34.309245 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:34.309161 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:35.309143 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:35.309103 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:35.309609 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:35.309157 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:35.309609 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:35.309254 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:35.309609 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:35.309394 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:36.309106 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:36.309068 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:36.309273 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:36.309197 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:37.309935 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:37.309899 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:37.309935 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:37.309913 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:37.310433 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:37.310201 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:37.310433 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:37.310339 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:37.893983 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:37.893945 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:37.893983 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:37.893999 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs\") pod \"network-metrics-daemon-j7bj7\" (UID: \"cfcce616-ce21-4462-865f-116401b2c78f\") " pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:37.894294 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:37.894148 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:37.894294 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:37.894234 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret podName:97b02d51-cfd3-4d11-a108-c54bf9dca12e nodeName:}" failed. No retries permitted until 2026-07-10 18:17:53.894211609 +0000 UTC m=+33.307496378 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret") pod "global-pull-secret-syncer-84n7b" (UID: "97b02d51-cfd3-4d11-a108-c54bf9dca12e") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:37.894294 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:37.894157 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:37.894450 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:37.894312 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs podName:cfcce616-ce21-4462-865f-116401b2c78f nodeName:}" failed. No retries permitted until 2026-07-10 18:17:53.89429171 +0000 UTC m=+33.307576475 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs") pod "network-metrics-daemon-j7bj7" (UID: "cfcce616-ce21-4462-865f-116401b2c78f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:37.995492 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:37.995436 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsjf5\" (UniqueName: \"kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5\") pod \"network-check-target-56zsp\" (UID: \"c453daa4-8e53-47b8-a056-7a7c32164c45\") " pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:37.995686 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:37.995622 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:17:37.995686 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:37.995642 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:17:37.995686 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:37.995655 2574 projected.go:196] Error preparing data for projected volume kube-api-access-tsjf5 for pod openshift-network-diagnostics/network-check-target-56zsp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:37.995829 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:37.995710 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5 podName:c453daa4-8e53-47b8-a056-7a7c32164c45 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:53.995692703 +0000 UTC m=+33.408977490 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tsjf5" (UniqueName: "kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5") pod "network-check-target-56zsp" (UID: "c453daa4-8e53-47b8-a056-7a7c32164c45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:38.309950 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:38.309872 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:38.310450 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:38.310004 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:39.309730 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:39.309695 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:39.309961 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:39.309709 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:39.309961 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:39.309840 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:39.309961 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:39.309920 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:40.309911 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:40.309727 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:40.310119 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:40.309995 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:41.310291 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:41.310245 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:41.310760 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:41.310381 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:41.310760 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:41.310420 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:41.310760 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:41.310483 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:42.309721 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.309486 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:42.309863 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:42.309832 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:42.452960 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.452911 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" event={"ID":"e9672184-d9a7-4913-9481-a10bbd1e7297","Type":"ContainerStarted","Data":"e67b984bccbdd1728336054d34a3cc81229a7c4a662ebdb2662fa324928b9797"} Jul 10 18:17:42.454662 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.454565 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7qhmx" event={"ID":"42571c86-4eaf-44a2-868e-1e023a349549","Type":"ContainerStarted","Data":"407b0782616994463acd330e4d9d08b2e9859f7d449a3b8ad6ed3da9c3ed3556"} Jul 10 18:17:42.456426 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.456397 2574 generic.go:356] "Generic (PLEG): container finished" podID="17653f02796255e2c412a08e496ac872" containerID="645a246c9b00f6667e5de80c4cfb6fb27749cb493469a7a9a8ff21404bc16d50" exitCode=0 Jul 10 18:17:42.456545 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.456478 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" event={"ID":"17653f02796255e2c412a08e496ac872","Type":"ContainerDied","Data":"645a246c9b00f6667e5de80c4cfb6fb27749cb493469a7a9a8ff21404bc16d50"} Jul 10 18:17:42.458394 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.458358 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" event={"ID":"7554e05d30e68919ee44413a4c1cb32c","Type":"ContainerStarted","Data":"ea7038f72b8ec0dd664970a32e2b6bb9c2a3dcdc7c553fb89888c1f0f2517d78"} Jul 10 18:17:42.460325 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.460296 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9rhl2" event={"ID":"952ce2e7-c21f-4fe0-9357-ced0cedef546","Type":"ContainerStarted","Data":"0ccdef75570141015a721c10a9e4ffcc51e5fc2e47d5b5ac00cf0955a3bf6996"} Jul 10 18:17:42.462623 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.462606 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:17:42.462963 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.462939 2574 generic.go:356] "Generic (PLEG): container finished" podID="0d031c2a-1702-4f4e-a90a-6e6767ebe663" containerID="ea4f7eb6ffeaecd0b6a20aed3e469ab1de31a60445e009e04b728fd374734cae" exitCode=1 Jul 10 18:17:42.463068 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.463014 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" event={"ID":"0d031c2a-1702-4f4e-a90a-6e6767ebe663","Type":"ContainerDied","Data":"ea4f7eb6ffeaecd0b6a20aed3e469ab1de31a60445e009e04b728fd374734cae"} Jul 10 18:17:42.463161 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.463111 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" event={"ID":"0d031c2a-1702-4f4e-a90a-6e6767ebe663","Type":"ContainerStarted","Data":"7e5f73675321b6a4007e374a79b900e9415e3a9d3b86838660e65d21300c9464"} Jul 10 18:17:42.464663 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.464633 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r98mv" event={"ID":"c2a28dd9-4966-4c3d-8c68-bfacfe569bea","Type":"ContainerStarted","Data":"12be527b28b1963948e4eb8b011ed9a75c67ccb2ce254fb0788e86fb8eaf4c24"} Jul 10 18:17:42.466200 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.466178 2574 generic.go:356] "Generic (PLEG): container finished" podID="a73fabd7-58c7-4f58-883c-1fd0a6650512" containerID="c1397a70c300b5791cfb6f509d8e1b14e53a37575ce66f116224f8c5e5898bc9" exitCode=0 Jul 10 18:17:42.466299 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.466251 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nj84h" event={"ID":"a73fabd7-58c7-4f58-883c-1fd0a6650512","Type":"ContainerDied","Data":"c1397a70c300b5791cfb6f509d8e1b14e53a37575ce66f116224f8c5e5898bc9"} Jul 10 18:17:42.467905 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.467834 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rl7np" event={"ID":"8cd76465-cc20-4ce3-b7c1-06c5bb9ee68a","Type":"ContainerStarted","Data":"e38e231cfe3dd476c41de27ac3e00151589b3a72f6e543c3c37ff149978101f0"} Jul 10 18:17:42.469427 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.469381 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7qhmx" podStartSLOduration=2.283480955 podStartE2EDuration="21.469370595s" totalImagesPullingTime="19.18588964s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:42.469122185 +0000 UTC m=+21.882406972" watchObservedRunningTime="2026-07-10 18:17:42.469370595 +0000 UTC m=+21.882655382" Jul 10 18:17:42.487249 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.486535 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-r98mv" podStartSLOduration=2.23526963 podStartE2EDuration="21.486517751s" totalImagesPullingTime="19.251248121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:42.486114439 +0000 UTC m=+21.899399221" watchObservedRunningTime="2026-07-10 18:17:42.486517751 +0000 UTC m=+21.899802541" Jul 10 18:17:42.517880 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.517826 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" podStartSLOduration=20.5178044 podStartE2EDuration="20.5178044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:42.517056052 +0000 UTC m=+21.930340840" watchObservedRunningTime="2026-07-10 18:17:42.5178044 +0000 UTC m=+21.931089188" Jul 10 18:17:42.518035 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.517911 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9rhl2" podStartSLOduration=2.363080105 podStartE2EDuration="21.517904797s" totalImagesPullingTime="19.154824692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:42.500130296 +0000 UTC m=+21.913415077" watchObservedRunningTime="2026-07-10 18:17:42.517904797 +0000 UTC m=+21.931189584" Jul 10 18:17:42.555125 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:42.554839 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rl7np" podStartSLOduration=2.433670433 podStartE2EDuration="21.554817318s" totalImagesPullingTime="19.121146885s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:42.553902435 +0000 UTC m=+21.967187223" watchObservedRunningTime="2026-07-10 18:17:42.554817318 +0000 UTC m=+21.968102108" Jul 10 18:17:43.309377 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:43.309346 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:43.309555 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:43.309346 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:43.309555 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:43.309479 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:43.309555 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:43.309523 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:43.471058 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:43.471020 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wrf88" event={"ID":"a315c03f-2c33-4fc6-8cb8-419a77746c77","Type":"ContainerStarted","Data":"fc3e98576bee3c19f359a0182bbbcb22ce21641356387bc71bedda7050ccb6e5"} Jul 10 18:17:43.472809 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:43.472786 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" event={"ID":"17653f02796255e2c412a08e496ac872","Type":"ContainerStarted","Data":"b5f1e6d31825f00552af0fe44c772654f477c6231aed3603c69078b02ea26747"} Jul 10 18:17:43.474965 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:43.474944 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:17:43.475396 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:43.475373 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" event={"ID":"0d031c2a-1702-4f4e-a90a-6e6767ebe663","Type":"ContainerStarted","Data":"599fee5a69a2b92de622b64245f86ad959471c77f8c252b7e5fcf8fde0f9dd2e"} Jul 10 18:17:43.475461 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:43.475448 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" event={"ID":"0d031c2a-1702-4f4e-a90a-6e6767ebe663","Type":"ContainerStarted","Data":"8cf2a38e0ac35aa8934114be6cb48a71f4a16ea9adab0f7ff7f2068744b7a3ed"} Jul 10 18:17:43.475512 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:43.475498 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" event={"ID":"0d031c2a-1702-4f4e-a90a-6e6767ebe663","Type":"ContainerStarted","Data":"54c23fd4f4c0ceacee135a04d8eef746022c2759d9f01594bee926dd43483680"} Jul 10 18:17:43.485887 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:43.485842 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wrf88" podStartSLOduration=3.284229958 podStartE2EDuration="22.485827279s" totalImagesPullingTime="19.201597321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:43.485560437 +0000 UTC m=+22.898845224" watchObservedRunningTime="2026-07-10 18:17:43.485827279 +0000 UTC m=+22.899112098" Jul 10 18:17:43.500068 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:43.499991 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" podStartSLOduration=21.499975221 podStartE2EDuration="21.499975221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:43.499751113 +0000 UTC m=+22.913035900" watchObservedRunningTime="2026-07-10 18:17:43.499975221 +0000 UTC m=+22.913260007" Jul 10 18:17:43.992202 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:43.992170 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 18:17:44.183744 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:44.183542 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T18:17:43.992190858Z","UUID":"0b27f3bc-ffea-4198-947e-3110347a2975","Handler":null,"Name":"","Endpoint":""} Jul 10 18:17:44.185144 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:44.185120 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 18:17:44.185288 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:44.185160 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 18:17:44.309110 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:44.309062 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:44.309268 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:44.309180 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:44.481708 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:44.481633 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:17:44.482439 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:44.482402 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" event={"ID":"0d031c2a-1702-4f4e-a90a-6e6767ebe663","Type":"ContainerStarted","Data":"238796389c99f756a639294948a6a3af7b2227485b906687c7ce10a8cee4388e"} Jul 10 18:17:44.484229 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:44.484201 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c8b4p" event={"ID":"1e02735c-c7f7-4070-9a1f-0410bace9cd9","Type":"ContainerStarted","Data":"524d1857ef7ef2414d7a8b9848363fd42bfa31dc4127a899bfb76ba6f82de734"} Jul 10 18:17:44.486893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:44.486867 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" event={"ID":"e9672184-d9a7-4913-9481-a10bbd1e7297","Type":"ContainerStarted","Data":"bc5f89f0a73328ac92e55c2461b8a337884f95191395aad7f6881cbe0ca46189"} Jul 10 18:17:44.499672 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:44.499622 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-c8b4p" podStartSLOduration=2.220088846 podStartE2EDuration="23.499599766s" totalImagesPullingTime="21.27951092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-10 18:17:44.498803803 +0000 UTC m=+23.912088675" watchObservedRunningTime="2026-07-10 18:17:44.499599766 +0000 UTC m=+23.912884554" Jul 10 18:17:44.760154 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:44.760060 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9rhl2" Jul 10 18:17:44.760795 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:44.760765 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9rhl2" Jul 10 18:17:45.309928 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:45.309893 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:45.310171 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:45.310051 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:45.310171 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:45.310120 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:45.310287 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:45.310250 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:45.491724 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:45.491519 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" event={"ID":"e9672184-d9a7-4913-9481-a10bbd1e7297","Type":"ContainerStarted","Data":"08831ca779f71f8aa4d4023360badcdfab6d0d4a472d5ae26f2c4d54c006d13f"} Jul 10 18:17:45.507861 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:45.507804 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wpd2" podStartSLOduration=2.392842942 podStartE2EDuration="24.507786117s" totalImagesPullingTime="22.114943175s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:45.507754946 +0000 UTC m=+24.921039737" watchObservedRunningTime="2026-07-10 18:17:45.507786117 +0000 UTC m=+24.921070883" Jul 10 18:17:46.309983 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:46.309887 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:46.310187 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:46.310033 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:46.497985 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:46.497945 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:17:46.498438 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:46.498387 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" event={"ID":"0d031c2a-1702-4f4e-a90a-6e6767ebe663","Type":"ContainerStarted","Data":"367aa86dc3e5f3a79b0a705e1481b7e1b74eaeae9ab6ede4d697140b84c56c17"} Jul 10 18:17:47.310087 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:47.310046 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:47.310283 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:47.310046 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:47.310283 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:47.310196 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:47.310283 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:47.310231 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:48.309602 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:48.309432 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:48.310026 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:48.309685 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:48.504675 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:48.504639 2574 generic.go:356] "Generic (PLEG): container finished" podID="a73fabd7-58c7-4f58-883c-1fd0a6650512" containerID="b8f1a292f17b42f031affefaf37d8a778109201b3df300060c46d05e55bc0a5b" exitCode=0 Jul 10 18:17:48.504833 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:48.504695 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nj84h" event={"ID":"a73fabd7-58c7-4f58-883c-1fd0a6650512","Type":"ContainerDied","Data":"b8f1a292f17b42f031affefaf37d8a778109201b3df300060c46d05e55bc0a5b"} Jul 10 18:17:48.508662 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:48.508639 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:17:48.509035 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:48.509014 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" event={"ID":"0d031c2a-1702-4f4e-a90a-6e6767ebe663","Type":"ContainerStarted","Data":"e15e5a9dae958776237e26f5947ca0fd0a6e3469c335054f7c2195df7320ec0e"} Jul 10 18:17:48.509297 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:48.509282 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:48.509376 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:48.509336 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:48.509474 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:48.509460 2574 scope.go:117] "RemoveContainer" containerID="ea4f7eb6ffeaecd0b6a20aed3e469ab1de31a60445e009e04b728fd374734cae" Jul 10 18:17:48.526537 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:48.526513 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:49.309641 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:49.309616 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:49.310276 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:49.309616 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:49.310276 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:49.309723 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:49.310276 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:49.309787 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:49.516621 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:49.516405 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:17:49.516978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:49.516951 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" event={"ID":"0d031c2a-1702-4f4e-a90a-6e6767ebe663","Type":"ContainerStarted","Data":"52bba7b2da3adf48a93d3a7f3d08852405570ef369f32c88f3d13e0a0e8795e3"} Jul 10 18:17:49.517400 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:49.517168 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:49.536733 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:49.536703 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:17:49.545154 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:49.545078 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" podStartSLOduration=8.909506159 podStartE2EDuration="28.545055634s" totalImagesPullingTime="19.635549475s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:49.545021708 +0000 UTC m=+28.958306496" watchObservedRunningTime="2026-07-10 18:17:49.545055634 +0000 UTC m=+28.958340419" Jul 10 18:17:50.265690 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:50.265657 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9rhl2" Jul 10 18:17:50.265909 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:50.265858 2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 18:17:50.266401 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:50.266375 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9rhl2" Jul 10 18:17:50.297812 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:50.295652 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-56zsp"] Jul 10 18:17:50.297812 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:50.295788 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:50.297812 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:50.295885 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:50.297812 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:50.296598 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j7bj7"] Jul 10 18:17:50.297812 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:50.296714 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:50.297812 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:50.296836 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:50.300144 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:50.300116 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-84n7b"] Jul 10 18:17:50.300275 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:50.300228 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:50.300333 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:50.300314 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:50.521223 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:50.521187 2574 generic.go:356] "Generic (PLEG): container finished" podID="a73fabd7-58c7-4f58-883c-1fd0a6650512" containerID="ee12e5bc0b650d6b52c581e537aa5a5003f294c93b594c1a028ca1e343ed42f7" exitCode=0 Jul 10 18:17:50.521633 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:50.521265 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nj84h" event={"ID":"a73fabd7-58c7-4f58-883c-1fd0a6650512","Type":"ContainerDied","Data":"ee12e5bc0b650d6b52c581e537aa5a5003f294c93b594c1a028ca1e343ed42f7"} Jul 10 18:17:52.309251 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:52.309004 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:52.309648 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:52.309035 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:52.309648 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:52.309271 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:52.309648 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:52.309363 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:52.309648 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:52.309038 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:52.309648 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:52.309449 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:52.528008 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:52.527977 2574 generic.go:356] "Generic (PLEG): container finished" podID="a73fabd7-58c7-4f58-883c-1fd0a6650512" containerID="a69ef9f6a10e2e28f03dd7138677f4b41d392f8e26d6698c97a49f5fa100e218" exitCode=0 Jul 10 18:17:52.528008 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:52.528025 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nj84h" event={"ID":"a73fabd7-58c7-4f58-883c-1fd0a6650512","Type":"ContainerDied","Data":"a69ef9f6a10e2e28f03dd7138677f4b41d392f8e26d6698c97a49f5fa100e218"} Jul 10 18:17:53.910749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:53.910710 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:53.910749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:53.910751 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs\") pod \"network-metrics-daemon-j7bj7\" (UID: \"cfcce616-ce21-4462-865f-116401b2c78f\") " pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:53.911198 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:53.910841 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:53.911198 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:53.910848 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:53.911198 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:53.910893 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs podName:cfcce616-ce21-4462-865f-116401b2c78f nodeName:}" failed. No retries permitted until 2026-07-10 18:18:25.91087855 +0000 UTC m=+65.324163319 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs") pod "network-metrics-daemon-j7bj7" (UID: "cfcce616-ce21-4462-865f-116401b2c78f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:53.911198 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:53.910906 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret podName:97b02d51-cfd3-4d11-a108-c54bf9dca12e nodeName:}" failed. No retries permitted until 2026-07-10 18:18:25.910899764 +0000 UTC m=+65.324184528 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret") pod "global-pull-secret-syncer-84n7b" (UID: "97b02d51-cfd3-4d11-a108-c54bf9dca12e") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:54.012025 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.011986 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsjf5\" (UniqueName: \"kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5\") pod \"network-check-target-56zsp\" (UID: \"c453daa4-8e53-47b8-a056-7a7c32164c45\") " pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:54.012212 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:54.012177 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:17:54.012212 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:54.012199 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:17:54.012212 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:54.012209 2574 projected.go:196] Error preparing data for projected volume kube-api-access-tsjf5 for pod openshift-network-diagnostics/network-check-target-56zsp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:54.012312 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:54.012266 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5 podName:c453daa4-8e53-47b8-a056-7a7c32164c45 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:26.012252743 +0000 UTC m=+65.425537508 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tsjf5" (UniqueName: "kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5") pod "network-check-target-56zsp" (UID: "c453daa4-8e53-47b8-a056-7a7c32164c45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:54.309465 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.309382 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:54.309465 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.309403 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:54.309465 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.309410 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:54.309732 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:54.309521 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84n7b" podUID="97b02d51-cfd3-4d11-a108-c54bf9dca12e" Jul 10 18:17:54.309732 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:54.309573 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-56zsp" podUID="c453daa4-8e53-47b8-a056-7a7c32164c45" Jul 10 18:17:54.309732 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:54.309642 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j7bj7" podUID="cfcce616-ce21-4462-865f-116401b2c78f" Jul 10 18:17:54.892489 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.892459 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeReady" Jul 10 18:17:54.892661 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.892591 2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 18:17:54.926362 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.926327 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7cb47549b8-x8zsn"] Jul 10 18:17:54.946835 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.946801 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh"] Jul 10 18:17:54.947004 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.946937 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:54.949929 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.949883 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6v2rr\"" Jul 10 18:17:54.950138 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.950081 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 18:17:54.950459 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.950442 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 18:17:54.950605 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.950588 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 18:17:54.959288 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.959072 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 18:17:54.967904 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.967881 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-b5h8k"] Jul 10 18:17:54.968029 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.968007 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" Jul 10 18:17:54.970595 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.970441 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-nzx9w\"" Jul 10 18:17:54.970595 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.970482 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 10 18:17:54.970595 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.970498 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:17:54.970792 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.970633 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 10 18:17:54.971019 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.971001 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 10 18:17:54.982975 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.982954 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz"] Jul 10 18:17:54.983138 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.983086 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:54.985576 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.985558 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 18:17:54.986199 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.986180 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 10 18:17:54.986736 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.986715 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 18:17:54.986901 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.986722 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 10 18:17:54.995266 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.995247 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qvxch\"" Jul 10 18:17:54.999201 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.999181 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-57f678b4d6-q2rhg"] Jul 10 18:17:54.999311 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:54.999292 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:17:55.015705 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.015681 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 10 18:17:55.016827 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.016802 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4sqzg\"" Jul 10 18:17:55.017211 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.017191 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 18:17:55.017460 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.017442 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 18:17:55.018561 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.018197 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-cjjrj"] Jul 10 18:17:55.018561 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.018318 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.023305 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.023249 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b5164ee-1735-42c2-8cf1-766f022b5703-trusted-ca\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.023402 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.023352 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-certificates\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.023402 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.023386 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6vlb\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-kube-api-access-z6vlb\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.023651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.023629 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b5164ee-1735-42c2-8cf1-766f022b5703-image-registry-private-configuration\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.023708 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.023671 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.023759 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.023707 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b5164ee-1735-42c2-8cf1-766f022b5703-installation-pull-secrets\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.023808 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.023778 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b5164ee-1735-42c2-8cf1-766f022b5703-ca-trust-extracted\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.023855 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.023810 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-bound-sa-token\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.025515 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.025488 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 10 18:17:55.025700 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.025675 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 10 18:17:55.026311 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.026294 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 10 18:17:55.027207 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.027188 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 18:17:55.027348 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.027329 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 10 18:17:55.027702 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.027684 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 10 18:17:55.027803 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.027781 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 10 18:17:55.028721 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.028684 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 18:17:55.030188 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.030025 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vm4x9\"" Jul 10 18:17:55.037134 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.037068 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn"] Jul 10 18:17:55.037272 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.037194 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:17:55.039970 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.039951 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:17:55.040073 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.039980 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 10 18:17:55.040272 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.040252 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 10 18:17:55.040872 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.040804 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 10 18:17:55.041396 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.041379 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-g64jl\"" Jul 10 18:17:55.055212 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.055188 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cb47549b8-x8zsn"] Jul 10 18:17:55.055339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.055222 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh"] Jul 10 18:17:55.055339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.055242 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz"] Jul 10 18:17:55.055339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.055258 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-b5h8k"] Jul 10 18:17:55.055339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.055276 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn"] Jul 10 18:17:55.055339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.055294 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-cjjrj"] Jul 10 18:17:55.055339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.055307 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57f678b4d6-q2rhg"] Jul 10 18:17:55.055339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.055321 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vp5ls"] Jul 10 18:17:55.055339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.055328 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" Jul 10 18:17:55.058398 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.058380 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-r2674\"" Jul 10 18:17:55.058587 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.058567 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 10 18:17:55.058786 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.058755 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 10 18:17:55.058940 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.058920 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:17:55.060466 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.060445 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 10 18:17:55.071713 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.071689 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq"] Jul 10 18:17:55.071826 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.071811 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vp5ls" Jul 10 18:17:55.074554 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.074532 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-28j2w\"" Jul 10 18:17:55.074854 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.074836 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 10 18:17:55.075310 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.075291 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:17:55.087053 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.087033 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-t89br"] Jul 10 18:17:55.087197 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.087171 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" Jul 10 18:17:55.089575 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.089552 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 10 18:17:55.089693 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.089662 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fvmq7\"" Jul 10 18:17:55.090149 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.090130 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 10 18:17:55.090254 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.090168 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 10 18:17:55.092918 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.092899 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:17:55.101412 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.101389 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq"] Jul 10 18:17:55.101521 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.101428 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vp5ls"] Jul 10 18:17:55.101521 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.101446 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-t89br"] Jul 10 18:17:55.101609 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.101529 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-t89br" Jul 10 18:17:55.101609 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.101550 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cpmcx"] Jul 10 18:17:55.104242 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.104226 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-g6xt8\"" Jul 10 18:17:55.104476 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.104463 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 18:17:55.104852 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.104837 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 18:17:55.119360 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.119331 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7nprf"] Jul 10 18:17:55.119509 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.119451 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cpmcx" Jul 10 18:17:55.122003 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.121971 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 18:17:55.122690 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.122672 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 18:17:55.123027 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.123011 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 18:17:55.123362 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.123342 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lzjfw\"" Jul 10 18:17:55.124383 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.124348 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b5164ee-1735-42c2-8cf1-766f022b5703-trusted-ca\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.124486 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.124386 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6vlb\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-kube-api-access-z6vlb\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.124486 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.124418 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-stats-auth\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.124486 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.124438 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5prd\" (UniqueName: \"kubernetes.io/projected/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-kube-api-access-f5prd\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.124486 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.124460 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e3770e0a-c127-454b-b0dc-6e4d4a9c1628-config\") pod \"console-operator-7866497494-cjjrj\" (UID: \"e3770e0a-c127-454b-b0dc-6e4d4a9c1628\") " pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:17:55.124486 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.124485 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/06f82976-1aae-4012-bac5-792e61e2c8af-snapshots\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.124721 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.124513 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.124721 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.124556 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx8cg\" (UniqueName: \"kubernetes.io/projected/20f79a38-b179-4205-bf87-71983574bfdd-kube-api-access-jx8cg\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:17:55.124721 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.124574 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f82976-1aae-4012-bac5-792e61e2c8af-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.124721 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.124603 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b5164ee-1735-42c2-8cf1-766f022b5703-ca-trust-extracted\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.125436 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.125216 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/20f79a38-b179-4205-bf87-71983574bfdd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:17:55.125436 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.125268 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-bound-sa-token\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.125436 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.125308 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-default-certificate\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.125436 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.125380 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b5164ee-1735-42c2-8cf1-766f022b5703-ca-trust-extracted\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.125715 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.125537 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.125715 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.125581 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/06f82976-1aae-4012-bac5-792e61e2c8af-serving-cert\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.125715 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.125622 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzf9c\" (UniqueName: \"kubernetes.io/projected/c81ab235-1f9d-41c8-af87-80ee3aa5c67c-kube-api-access-kzf9c\") pod \"service-ca-operator-7695fc6c8c-7dmkh\" (UID: \"c81ab235-1f9d-41c8-af87-80ee3aa5c67c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" Jul 10 18:17:55.125715 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.125702 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f82976-1aae-4012-bac5-792e61e2c8af-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.125903 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.125799 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e3770e0a-c127-454b-b0dc-6e4d4a9c1628-serving-cert\") pod \"console-operator-7866497494-cjjrj\" (UID: \"e3770e0a-c127-454b-b0dc-6e4d4a9c1628\") " pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:17:55.125903 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.125894 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-certificates\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.126006 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.125916 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vllwr\" (UniqueName: \"kubernetes.io/projected/e3770e0a-c127-454b-b0dc-6e4d4a9c1628-kube-api-access-vllwr\") pod \"console-operator-7866497494-cjjrj\" (UID: \"e3770e0a-c127-454b-b0dc-6e4d4a9c1628\") " pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:17:55.126775 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.126058 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b5164ee-1735-42c2-8cf1-766f022b5703-image-registry-private-configuration\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.126775 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.126389 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b5164ee-1735-42c2-8cf1-766f022b5703-trusted-ca\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.126775 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.126456 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.126775 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.126468 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-certificates\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.126775 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.126499 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b5164ee-1735-42c2-8cf1-766f022b5703-installation-pull-secrets\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.126775 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.126533 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:17:55.126775 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.126567 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06f82976-1aae-4012-bac5-792e61e2c8af-tmp\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.126775 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.126593 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:17:55.126775 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.126604 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5786x\" (UniqueName: \"kubernetes.io/projected/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-kube-api-access-5786x\") pod \"cluster-samples-operator-65db858894-cqgzn\" (UID: \"ae73be3c-7b12-4e96-9afb-08b5e3f10c99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" Jul 10 18:17:55.126775 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.126613 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cb47549b8-x8zsn: secret "image-registry-tls" not found Jul 10 18:17:55.126775 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.126668 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-cqgzn\" (UID: \"ae73be3c-7b12-4e96-9afb-08b5e3f10c99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" Jul 10 18:17:55.126775 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.126667 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls podName:8b5164ee-1735-42c2-8cf1-766f022b5703 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:55.626646342 +0000 UTC m=+35.039931121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls") pod "image-registry-7cb47549b8-x8zsn" (UID: "8b5164ee-1735-42c2-8cf1-766f022b5703") : secret "image-registry-tls" not found Jul 10 18:17:55.126775 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.126778 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtn7p\" (UniqueName: \"kubernetes.io/projected/06f82976-1aae-4012-bac5-792e61e2c8af-kube-api-access-xtn7p\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.127953 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.126809 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c81ab235-1f9d-41c8-af87-80ee3aa5c67c-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7dmkh\" (UID: \"c81ab235-1f9d-41c8-af87-80ee3aa5c67c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" Jul 10 18:17:55.127953 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.126871 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3770e0a-c127-454b-b0dc-6e4d4a9c1628-trusted-ca\") pod \"console-operator-7866497494-cjjrj\" (UID: \"e3770e0a-c127-454b-b0dc-6e4d4a9c1628\") " pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:17:55.127953 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.126940 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c81ab235-1f9d-41c8-af87-80ee3aa5c67c-config\") pod \"service-ca-operator-7695fc6c8c-7dmkh\" (UID: \"c81ab235-1f9d-41c8-af87-80ee3aa5c67c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" Jul 10 18:17:55.131745 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.131711 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cpmcx"] Jul 10 18:17:55.131831 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.131758 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7nprf"] Jul 10 18:17:55.131952 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.131934 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:55.133828 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.133806 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b5164ee-1735-42c2-8cf1-766f022b5703-installation-pull-secrets\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.133991 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.133919 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b5164ee-1735-42c2-8cf1-766f022b5703-image-registry-private-configuration\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.134633 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.134609 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 18:17:55.134739 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.134661 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 18:17:55.134918 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.134901 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p5qqq\"" Jul 10 18:17:55.139001 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.138977 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6vlb\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-kube-api-access-z6vlb\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.142045 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.142026 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-bound-sa-token\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.227834 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.227794 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d6ade376-624c-4dfc-858b-1940181dfbb3-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-7x8dq\" (UID: \"d6ade376-624c-4dfc-858b-1940181dfbb3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" Jul 10 18:17:55.228014 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.227849 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx8cg\" (UniqueName: \"kubernetes.io/projected/20f79a38-b179-4205-bf87-71983574bfdd-kube-api-access-jx8cg\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:17:55.228014 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.227900 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f82976-1aae-4012-bac5-792e61e2c8af-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.228014 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.227928 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6ade376-624c-4dfc-858b-1940181dfbb3-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-7x8dq\" (UID: \"d6ade376-624c-4dfc-858b-1940181dfbb3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" Jul 10 18:17:55.228014 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.227958 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/20f79a38-b179-4205-bf87-71983574bfdd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:17:55.228014 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.227994 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d29817f3-db17-4d2b-9666-20e677724b75-config-volume\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:55.228014 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228012 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d29817f3-db17-4d2b-9666-20e677724b75-tmp-dir\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:55.228358 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228039 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-default-certificate\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.228358 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228066 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.228358 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228083 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/06f82976-1aae-4012-bac5-792e61e2c8af-serving-cert\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.228358 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228128 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzf9c\" (UniqueName: \"kubernetes.io/projected/c81ab235-1f9d-41c8-af87-80ee3aa5c67c-kube-api-access-kzf9c\") pod \"service-ca-operator-7695fc6c8c-7dmkh\" (UID: \"c81ab235-1f9d-41c8-af87-80ee3aa5c67c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" Jul 10 18:17:55.228358 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228155 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f82976-1aae-4012-bac5-792e61e2c8af-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.228358 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228178 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60217543-943f-4011-8a09-132f51157b28-cert\") pod \"ingress-canary-cpmcx\" (UID: \"60217543-943f-4011-8a09-132f51157b28\") " pod="openshift-ingress-canary/ingress-canary-cpmcx" Jul 10 18:17:55.228358 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228201 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:55.228358 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.228233 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle podName:b8a376b5-b2dc-439a-aa9c-b1bd4f37252d nodeName:}" failed. No retries permitted until 2026-07-10 18:17:55.728212341 +0000 UTC m=+35.141497123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle") pod "router-default-57f678b4d6-q2rhg" (UID: "b8a376b5-b2dc-439a-aa9c-b1bd4f37252d") : configmap references non-existent config key: service-ca.crt Jul 10 18:17:55.228358 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228277 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwp7c\" (UniqueName: \"kubernetes.io/projected/e953a150-6569-49ea-8f29-fc6503cf328a-kube-api-access-bwp7c\") pod \"volume-data-source-validator-78495597f6-vp5ls\" (UID: \"e953a150-6569-49ea-8f29-fc6503cf328a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vp5ls" Jul 10 18:17:55.228358 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228309 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e3770e0a-c127-454b-b0dc-6e4d4a9c1628-serving-cert\") pod \"console-operator-7866497494-cjjrj\" (UID: \"e3770e0a-c127-454b-b0dc-6e4d4a9c1628\") " pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:17:55.228358 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228345 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vllwr\" (UniqueName: \"kubernetes.io/projected/e3770e0a-c127-454b-b0dc-6e4d4a9c1628-kube-api-access-vllwr\") pod \"console-operator-7866497494-cjjrj\" (UID: \"e3770e0a-c127-454b-b0dc-6e4d4a9c1628\") " pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:17:55.228893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228388 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmf6v\" (UniqueName: \"kubernetes.io/projected/60217543-943f-4011-8a09-132f51157b28-kube-api-access-tmf6v\") pod \"ingress-canary-cpmcx\" (UID: \"60217543-943f-4011-8a09-132f51157b28\") " pod="openshift-ingress-canary/ingress-canary-cpmcx" Jul 10 18:17:55.228893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228411 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f2d6\" (UniqueName: \"kubernetes.io/projected/d6ade376-624c-4dfc-858b-1940181dfbb3-kube-api-access-6f2d6\") pod \"kube-storage-version-migrator-operator-799cb7f958-7x8dq\" (UID: \"d6ade376-624c-4dfc-858b-1940181dfbb3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" Jul 10 18:17:55.228893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228479 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:17:55.228893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228514 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06f82976-1aae-4012-bac5-792e61e2c8af-tmp\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.228893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228544 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5786x\" (UniqueName: \"kubernetes.io/projected/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-kube-api-access-5786x\") pod \"cluster-samples-operator-65db858894-cqgzn\" (UID: \"ae73be3c-7b12-4e96-9afb-08b5e3f10c99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" Jul 10 18:17:55.228893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228571 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-cqgzn\" (UID: \"ae73be3c-7b12-4e96-9afb-08b5e3f10c99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" Jul 10 18:17:55.228893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228596 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtn7p\" (UniqueName: \"kubernetes.io/projected/06f82976-1aae-4012-bac5-792e61e2c8af-kube-api-access-xtn7p\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.228893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228621 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj82k\" (UniqueName: \"kubernetes.io/projected/d29817f3-db17-4d2b-9666-20e677724b75-kube-api-access-hj82k\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:55.228893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228654 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c81ab235-1f9d-41c8-af87-80ee3aa5c67c-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7dmkh\" (UID: \"c81ab235-1f9d-41c8-af87-80ee3aa5c67c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" Jul 10 18:17:55.228893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228684 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3770e0a-c127-454b-b0dc-6e4d4a9c1628-trusted-ca\") pod \"console-operator-7866497494-cjjrj\" (UID: \"e3770e0a-c127-454b-b0dc-6e4d4a9c1628\") " pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:17:55.228893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228712 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c81ab235-1f9d-41c8-af87-80ee3aa5c67c-config\") pod \"service-ca-operator-7695fc6c8c-7dmkh\" (UID: \"c81ab235-1f9d-41c8-af87-80ee3aa5c67c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" Jul 10 18:17:55.228893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228771 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-stats-auth\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.228893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228803 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kz8m\" (UniqueName: \"kubernetes.io/projected/d8158185-b1f6-45a7-af6d-458b47d5dd8e-kube-api-access-9kz8m\") pod \"network-check-source-869dcfcdcd-t89br\" (UID: \"d8158185-b1f6-45a7-af6d-458b47d5dd8e\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-t89br" Jul 10 18:17:55.228893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228897 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5prd\" (UniqueName: \"kubernetes.io/projected/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-kube-api-access-f5prd\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.229484 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228932 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e3770e0a-c127-454b-b0dc-6e4d4a9c1628-config\") pod \"console-operator-7866497494-cjjrj\" (UID: \"e3770e0a-c127-454b-b0dc-6e4d4a9c1628\") " pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:17:55.229484 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228957 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f82976-1aae-4012-bac5-792e61e2c8af-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.229484 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228965 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/06f82976-1aae-4012-bac5-792e61e2c8af-snapshots\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.229484 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228773 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/20f79a38-b179-4205-bf87-71983574bfdd-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:17:55.229484 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228994 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.229484 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.229070 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:17:55.229484 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.229140 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs podName:b8a376b5-b2dc-439a-aa9c-b1bd4f37252d nodeName:}" failed. No retries permitted until 2026-07-10 18:17:55.729122652 +0000 UTC m=+35.142407435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs") pod "router-default-57f678b4d6-q2rhg" (UID: "b8a376b5-b2dc-439a-aa9c-b1bd4f37252d") : secret "router-metrics-certs-default" not found Jul 10 18:17:55.229484 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.229138 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:17:55.229484 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.229179 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls podName:ae73be3c-7b12-4e96-9afb-08b5e3f10c99 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:55.729169148 +0000 UTC m=+35.142453913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls") pod "cluster-samples-operator-65db858894-cqgzn" (UID: "ae73be3c-7b12-4e96-9afb-08b5e3f10c99") : secret "samples-operator-tls" not found Jul 10 18:17:55.229484 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.228965 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f82976-1aae-4012-bac5-792e61e2c8af-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.229484 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.229283 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:17:55.229484 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.229298 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06f82976-1aae-4012-bac5-792e61e2c8af-tmp\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.229484 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.229320 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls podName:20f79a38-b179-4205-bf87-71983574bfdd nodeName:}" failed. No retries permitted until 2026-07-10 18:17:55.729308104 +0000 UTC m=+35.142592871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-scpbz" (UID: "20f79a38-b179-4205-bf87-71983574bfdd") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:17:55.230141 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.229686 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/06f82976-1aae-4012-bac5-792e61e2c8af-snapshots\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.230141 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.229876 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c81ab235-1f9d-41c8-af87-80ee3aa5c67c-config\") pod \"service-ca-operator-7695fc6c8c-7dmkh\" (UID: \"c81ab235-1f9d-41c8-af87-80ee3aa5c67c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" Jul 10 18:17:55.230349 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.230294 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e3770e0a-c127-454b-b0dc-6e4d4a9c1628-config\") pod \"console-operator-7866497494-cjjrj\" (UID: \"e3770e0a-c127-454b-b0dc-6e4d4a9c1628\") " pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:17:55.230998 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.230570 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3770e0a-c127-454b-b0dc-6e4d4a9c1628-trusted-ca\") pod \"console-operator-7866497494-cjjrj\" (UID: \"e3770e0a-c127-454b-b0dc-6e4d4a9c1628\") " pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:17:55.230998 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.230974 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/06f82976-1aae-4012-bac5-792e61e2c8af-serving-cert\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.231441 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.231413 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c81ab235-1f9d-41c8-af87-80ee3aa5c67c-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7dmkh\" (UID: \"c81ab235-1f9d-41c8-af87-80ee3aa5c67c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" Jul 10 18:17:55.231566 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.231551 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-default-certificate\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.231927 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.231909 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-stats-auth\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.232082 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.232064 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e3770e0a-c127-454b-b0dc-6e4d4a9c1628-serving-cert\") pod \"console-operator-7866497494-cjjrj\" (UID: \"e3770e0a-c127-454b-b0dc-6e4d4a9c1628\") " pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:17:55.243036 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.242981 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzf9c\" (UniqueName: \"kubernetes.io/projected/c81ab235-1f9d-41c8-af87-80ee3aa5c67c-kube-api-access-kzf9c\") pod \"service-ca-operator-7695fc6c8c-7dmkh\" (UID: \"c81ab235-1f9d-41c8-af87-80ee3aa5c67c\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" Jul 10 18:17:55.243301 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.243278 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtn7p\" (UniqueName: \"kubernetes.io/projected/06f82976-1aae-4012-bac5-792e61e2c8af-kube-api-access-xtn7p\") pod \"insights-operator-68d9dd94cf-b5h8k\" (UID: \"06f82976-1aae-4012-bac5-792e61e2c8af\") " pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.243622 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.243599 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5786x\" (UniqueName: \"kubernetes.io/projected/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-kube-api-access-5786x\") pod \"cluster-samples-operator-65db858894-cqgzn\" (UID: \"ae73be3c-7b12-4e96-9afb-08b5e3f10c99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" Jul 10 18:17:55.243678 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.243631 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vllwr\" (UniqueName: \"kubernetes.io/projected/e3770e0a-c127-454b-b0dc-6e4d4a9c1628-kube-api-access-vllwr\") pod \"console-operator-7866497494-cjjrj\" (UID: \"e3770e0a-c127-454b-b0dc-6e4d4a9c1628\") " pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:17:55.243756 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.243742 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5prd\" (UniqueName: \"kubernetes.io/projected/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-kube-api-access-f5prd\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.244062 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.244046 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx8cg\" (UniqueName: \"kubernetes.io/projected/20f79a38-b179-4205-bf87-71983574bfdd-kube-api-access-jx8cg\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:17:55.279848 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.279814 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" Jul 10 18:17:55.292983 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.292942 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" Jul 10 18:17:55.330340 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.330309 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmf6v\" (UniqueName: \"kubernetes.io/projected/60217543-943f-4011-8a09-132f51157b28-kube-api-access-tmf6v\") pod \"ingress-canary-cpmcx\" (UID: \"60217543-943f-4011-8a09-132f51157b28\") " pod="openshift-ingress-canary/ingress-canary-cpmcx" Jul 10 18:17:55.330478 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.330353 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f2d6\" (UniqueName: \"kubernetes.io/projected/d6ade376-624c-4dfc-858b-1940181dfbb3-kube-api-access-6f2d6\") pod \"kube-storage-version-migrator-operator-799cb7f958-7x8dq\" (UID: \"d6ade376-624c-4dfc-858b-1940181dfbb3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" Jul 10 18:17:55.330478 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.330435 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj82k\" (UniqueName: \"kubernetes.io/projected/d29817f3-db17-4d2b-9666-20e677724b75-kube-api-access-hj82k\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:55.330586 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.330488 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kz8m\" (UniqueName: \"kubernetes.io/projected/d8158185-b1f6-45a7-af6d-458b47d5dd8e-kube-api-access-9kz8m\") pod \"network-check-source-869dcfcdcd-t89br\" (UID: \"d8158185-b1f6-45a7-af6d-458b47d5dd8e\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-t89br" Jul 10 18:17:55.330586 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.330537 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d6ade376-624c-4dfc-858b-1940181dfbb3-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-7x8dq\" (UID: \"d6ade376-624c-4dfc-858b-1940181dfbb3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" Jul 10 18:17:55.330736 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.330694 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6ade376-624c-4dfc-858b-1940181dfbb3-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-7x8dq\" (UID: \"d6ade376-624c-4dfc-858b-1940181dfbb3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" Jul 10 18:17:55.330820 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.330750 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d29817f3-db17-4d2b-9666-20e677724b75-config-volume\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:55.330820 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.330771 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d29817f3-db17-4d2b-9666-20e677724b75-tmp-dir\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:55.330928 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.330873 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60217543-943f-4011-8a09-132f51157b28-cert\") pod \"ingress-canary-cpmcx\" (UID: \"60217543-943f-4011-8a09-132f51157b28\") " pod="openshift-ingress-canary/ingress-canary-cpmcx" Jul 10 18:17:55.330928 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.330906 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:55.331022 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.330943 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwp7c\" (UniqueName: \"kubernetes.io/projected/e953a150-6569-49ea-8f29-fc6503cf328a-kube-api-access-bwp7c\") pod \"volume-data-source-validator-78495597f6-vp5ls\" (UID: \"e953a150-6569-49ea-8f29-fc6503cf328a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vp5ls" Jul 10 18:17:55.331202 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.331179 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d29817f3-db17-4d2b-9666-20e677724b75-tmp-dir\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:55.331467 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.331292 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:17:55.331467 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.331347 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls podName:d29817f3-db17-4d2b-9666-20e677724b75 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:55.831329832 +0000 UTC m=+35.244614614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls") pod "dns-default-7nprf" (UID: "d29817f3-db17-4d2b-9666-20e677724b75") : secret "dns-default-metrics-tls" not found Jul 10 18:17:55.331467 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.331364 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6ade376-624c-4dfc-858b-1940181dfbb3-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-7x8dq\" (UID: \"d6ade376-624c-4dfc-858b-1940181dfbb3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" Jul 10 18:17:55.331467 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.331394 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d29817f3-db17-4d2b-9666-20e677724b75-config-volume\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:55.332941 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.332919 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d6ade376-624c-4dfc-858b-1940181dfbb3-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-7x8dq\" (UID: \"d6ade376-624c-4dfc-858b-1940181dfbb3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" Jul 10 18:17:55.333291 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.333271 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60217543-943f-4011-8a09-132f51157b28-cert\") pod \"ingress-canary-cpmcx\" (UID: \"60217543-943f-4011-8a09-132f51157b28\") " pod="openshift-ingress-canary/ingress-canary-cpmcx" Jul 10 18:17:55.340203 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.340169 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmf6v\" (UniqueName: \"kubernetes.io/projected/60217543-943f-4011-8a09-132f51157b28-kube-api-access-tmf6v\") pod \"ingress-canary-cpmcx\" (UID: \"60217543-943f-4011-8a09-132f51157b28\") " pod="openshift-ingress-canary/ingress-canary-cpmcx" Jul 10 18:17:55.340776 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.340754 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f2d6\" (UniqueName: \"kubernetes.io/projected/d6ade376-624c-4dfc-858b-1940181dfbb3-kube-api-access-6f2d6\") pod \"kube-storage-version-migrator-operator-799cb7f958-7x8dq\" (UID: \"d6ade376-624c-4dfc-858b-1940181dfbb3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" Jul 10 18:17:55.340887 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.340792 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj82k\" (UniqueName: \"kubernetes.io/projected/d29817f3-db17-4d2b-9666-20e677724b75-kube-api-access-hj82k\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:55.341124 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.341055 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kz8m\" (UniqueName: \"kubernetes.io/projected/d8158185-b1f6-45a7-af6d-458b47d5dd8e-kube-api-access-9kz8m\") pod \"network-check-source-869dcfcdcd-t89br\" (UID: \"d8158185-b1f6-45a7-af6d-458b47d5dd8e\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-t89br" Jul 10 18:17:55.341215 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.341125 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwp7c\" (UniqueName: \"kubernetes.io/projected/e953a150-6569-49ea-8f29-fc6503cf328a-kube-api-access-bwp7c\") pod \"volume-data-source-validator-78495597f6-vp5ls\" (UID: \"e953a150-6569-49ea-8f29-fc6503cf328a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vp5ls" Jul 10 18:17:55.347754 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.347731 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:17:55.381380 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.381349 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vp5ls" Jul 10 18:17:55.398233 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.398193 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" Jul 10 18:17:55.411838 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.411780 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-t89br" Jul 10 18:17:55.431947 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.431923 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cpmcx" Jul 10 18:17:55.634276 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.634231 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:55.634440 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.634390 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:17:55.634440 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.634410 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cb47549b8-x8zsn: secret "image-registry-tls" not found Jul 10 18:17:55.634530 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.634482 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls podName:8b5164ee-1735-42c2-8cf1-766f022b5703 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:56.634464305 +0000 UTC m=+36.047749075 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls") pod "image-registry-7cb47549b8-x8zsn" (UID: "8b5164ee-1735-42c2-8cf1-766f022b5703") : secret "image-registry-tls" not found Jul 10 18:17:55.734969 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.734887 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.734969 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.734956 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:55.735198 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.735051 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:17:55.735198 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.735051 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:17:55.735198 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.735130 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:17:55.735198 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.735131 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle podName:b8a376b5-b2dc-439a-aa9c-b1bd4f37252d nodeName:}" failed. No retries permitted until 2026-07-10 18:17:56.735107543 +0000 UTC m=+36.148392328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle") pod "router-default-57f678b4d6-q2rhg" (UID: "b8a376b5-b2dc-439a-aa9c-b1bd4f37252d") : configmap references non-existent config key: service-ca.crt Jul 10 18:17:55.735198 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.735190 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs podName:b8a376b5-b2dc-439a-aa9c-b1bd4f37252d nodeName:}" failed. No retries permitted until 2026-07-10 18:17:56.735174901 +0000 UTC m=+36.148459680 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs") pod "router-default-57f678b4d6-q2rhg" (UID: "b8a376b5-b2dc-439a-aa9c-b1bd4f37252d") : secret "router-metrics-certs-default" not found Jul 10 18:17:55.735413 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.735210 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls podName:20f79a38-b179-4205-bf87-71983574bfdd nodeName:}" failed. No retries permitted until 2026-07-10 18:17:56.735199158 +0000 UTC m=+36.148483928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-scpbz" (UID: "20f79a38-b179-4205-bf87-71983574bfdd") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:17:55.735413 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.735253 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-cqgzn\" (UID: \"ae73be3c-7b12-4e96-9afb-08b5e3f10c99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" Jul 10 18:17:55.735413 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.735314 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:17:55.735413 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.735351 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls podName:ae73be3c-7b12-4e96-9afb-08b5e3f10c99 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:56.735343116 +0000 UTC m=+36.148627884 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls") pod "cluster-samples-operator-65db858894-cqgzn" (UID: "ae73be3c-7b12-4e96-9afb-08b5e3f10c99") : secret "samples-operator-tls" not found Jul 10 18:17:55.835973 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:55.835941 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:55.836162 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.836081 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:17:55.836222 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:55.836167 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls podName:d29817f3-db17-4d2b-9666-20e677724b75 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:56.836144435 +0000 UTC m=+36.249429199 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls") pod "dns-default-7nprf" (UID: "d29817f3-db17-4d2b-9666-20e677724b75") : secret "dns-default-metrics-tls" not found Jul 10 18:17:56.309505 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:56.309467 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:17:56.310149 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:56.309517 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:17:56.310149 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:56.309476 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:17:56.312509 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:56.312488 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 18:17:56.312509 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:56.312508 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 18:17:56.313371 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:56.313354 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nsh8q\"" Jul 10 18:17:56.313459 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:56.313402 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xs98z\"" Jul 10 18:17:56.643431 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:56.643337 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:56.643580 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:56.643501 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:17:56.643580 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:56.643522 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cb47549b8-x8zsn: secret "image-registry-tls" not found Jul 10 18:17:56.643580 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:56.643581 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls podName:8b5164ee-1735-42c2-8cf1-766f022b5703 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:58.643562732 +0000 UTC m=+38.056847520 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls") pod "image-registry-7cb47549b8-x8zsn" (UID: "8b5164ee-1735-42c2-8cf1-766f022b5703") : secret "image-registry-tls" not found Jul 10 18:17:56.744167 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:56.744123 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:17:56.744354 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:56.744186 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-cqgzn\" (UID: \"ae73be3c-7b12-4e96-9afb-08b5e3f10c99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" Jul 10 18:17:56.744354 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:56.744254 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:56.744354 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:56.744322 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:17:56.744354 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:56.744335 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:17:56.744567 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:56.744360 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:56.744567 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:56.744390 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls podName:20f79a38-b179-4205-bf87-71983574bfdd nodeName:}" failed. No retries permitted until 2026-07-10 18:17:58.744370136 +0000 UTC m=+38.157654925 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-scpbz" (UID: "20f79a38-b179-4205-bf87-71983574bfdd") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:17:56.744567 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:56.744398 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:17:56.744567 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:56.744429 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls podName:ae73be3c-7b12-4e96-9afb-08b5e3f10c99 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:58.744417096 +0000 UTC m=+38.157701864 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls") pod "cluster-samples-operator-65db858894-cqgzn" (UID: "ae73be3c-7b12-4e96-9afb-08b5e3f10c99") : secret "samples-operator-tls" not found Jul 10 18:17:56.744567 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:56.744444 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs podName:b8a376b5-b2dc-439a-aa9c-b1bd4f37252d nodeName:}" failed. No retries permitted until 2026-07-10 18:17:58.744436089 +0000 UTC m=+38.157720866 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs") pod "router-default-57f678b4d6-q2rhg" (UID: "b8a376b5-b2dc-439a-aa9c-b1bd4f37252d") : secret "router-metrics-certs-default" not found Jul 10 18:17:56.744567 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:56.744543 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle podName:b8a376b5-b2dc-439a-aa9c-b1bd4f37252d nodeName:}" failed. No retries permitted until 2026-07-10 18:17:58.744520079 +0000 UTC m=+38.157804864 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle") pod "router-default-57f678b4d6-q2rhg" (UID: "b8a376b5-b2dc-439a-aa9c-b1bd4f37252d") : configmap references non-existent config key: service-ca.crt Jul 10 18:17:56.847693 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:56.847186 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:56.848484 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:56.848439 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:17:56.849233 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:56.849161 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls podName:d29817f3-db17-4d2b-9666-20e677724b75 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:58.84913362 +0000 UTC m=+38.262418390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls") pod "dns-default-7nprf" (UID: "d29817f3-db17-4d2b-9666-20e677724b75") : secret "dns-default-metrics-tls" not found Jul 10 18:17:57.064301 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.063994 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-cjjrj"] Jul 10 18:17:57.070306 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.070282 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vp5ls"] Jul 10 18:17:57.075775 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.075748 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh"] Jul 10 18:17:57.087898 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.087867 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq"] Jul 10 18:17:57.091593 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.091562 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-t89br"] Jul 10 18:17:57.092494 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.092442 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cpmcx"] Jul 10 18:17:57.093279 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.093207 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-b5h8k"] Jul 10 18:17:57.115561 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:57.115517 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3770e0a_c127_454b_b0dc_6e4d4a9c1628.slice/crio-daa5d3e9e32fbd4f0c8dc7b05c18863ff3a4ee2faed268d04229c3316f659d67 WatchSource:0}: Error finding container daa5d3e9e32fbd4f0c8dc7b05c18863ff3a4ee2faed268d04229c3316f659d67: Status 404 returned error can't find the container with id daa5d3e9e32fbd4f0c8dc7b05c18863ff3a4ee2faed268d04229c3316f659d67 Jul 10 18:17:57.115973 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:57.115948 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode953a150_6569_49ea_8f29_fc6503cf328a.slice/crio-7acba0eaae2214908168c153a82dfe83ef6aaf27cf1b5f7c7401f21c9c4aba47 WatchSource:0}: Error finding container 7acba0eaae2214908168c153a82dfe83ef6aaf27cf1b5f7c7401f21c9c4aba47: Status 404 returned error can't find the container with id 7acba0eaae2214908168c153a82dfe83ef6aaf27cf1b5f7c7401f21c9c4aba47 Jul 10 18:17:57.116888 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:57.116864 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc81ab235_1f9d_41c8_af87_80ee3aa5c67c.slice/crio-d3e55fbf796124f088d9c9705667cf71473aaa4a29437031a3568920b077ebf2 WatchSource:0}: Error finding container d3e55fbf796124f088d9c9705667cf71473aaa4a29437031a3568920b077ebf2: Status 404 returned error can't find the container with id d3e55fbf796124f088d9c9705667cf71473aaa4a29437031a3568920b077ebf2 Jul 10 18:17:57.118636 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:57.118019 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6ade376_624c_4dfc_858b_1940181dfbb3.slice/crio-5635a9e92450c28c2bc045ce89b0cffa39763011a701e80caa5255af44270b2d WatchSource:0}: Error finding container 5635a9e92450c28c2bc045ce89b0cffa39763011a701e80caa5255af44270b2d: Status 404 returned error can't find the container with id 5635a9e92450c28c2bc045ce89b0cffa39763011a701e80caa5255af44270b2d Jul 10 18:17:57.119195 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:57.119157 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06f82976_1aae_4012_bac5_792e61e2c8af.slice/crio-89b894951043812006120664f9bd6337eff27b2bea4807a78a75158e4f06bff3 WatchSource:0}: Error finding container 89b894951043812006120664f9bd6337eff27b2bea4807a78a75158e4f06bff3: Status 404 returned error can't find the container with id 89b894951043812006120664f9bd6337eff27b2bea4807a78a75158e4f06bff3 Jul 10 18:17:57.119954 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:17:57.119854 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60217543_943f_4011_8a09_132f51157b28.slice/crio-12760ba501e2fe8a41ee870bc73695ef769acbc8a527ac71368429cc754cf8aa WatchSource:0}: Error finding container 12760ba501e2fe8a41ee870bc73695ef769acbc8a527ac71368429cc754cf8aa: Status 404 returned error can't find the container with id 12760ba501e2fe8a41ee870bc73695ef769acbc8a527ac71368429cc754cf8aa Jul 10 18:17:57.542303 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.542197 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vp5ls" event={"ID":"e953a150-6569-49ea-8f29-fc6503cf328a","Type":"ContainerStarted","Data":"7acba0eaae2214908168c153a82dfe83ef6aaf27cf1b5f7c7401f21c9c4aba47"} Jul 10 18:17:57.543473 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.543443 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" event={"ID":"06f82976-1aae-4012-bac5-792e61e2c8af","Type":"ContainerStarted","Data":"89b894951043812006120664f9bd6337eff27b2bea4807a78a75158e4f06bff3"} Jul 10 18:17:57.544718 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.544696 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" event={"ID":"d6ade376-624c-4dfc-858b-1940181dfbb3","Type":"ContainerStarted","Data":"5635a9e92450c28c2bc045ce89b0cffa39763011a701e80caa5255af44270b2d"} Jul 10 18:17:57.550656 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.550618 2574 generic.go:356] "Generic (PLEG): container finished" podID="a73fabd7-58c7-4f58-883c-1fd0a6650512" containerID="d3345a92079994b081d186f357f692b8c5a4c8d85b6b5cf806af2b0678fbd702" exitCode=0 Jul 10 18:17:57.550656 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.550653 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nj84h" event={"ID":"a73fabd7-58c7-4f58-883c-1fd0a6650512","Type":"ContainerDied","Data":"d3345a92079994b081d186f357f692b8c5a4c8d85b6b5cf806af2b0678fbd702"} Jul 10 18:17:57.552054 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.552028 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cpmcx" event={"ID":"60217543-943f-4011-8a09-132f51157b28","Type":"ContainerStarted","Data":"12760ba501e2fe8a41ee870bc73695ef769acbc8a527ac71368429cc754cf8aa"} Jul 10 18:17:57.553156 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.553131 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-cjjrj" event={"ID":"e3770e0a-c127-454b-b0dc-6e4d4a9c1628","Type":"ContainerStarted","Data":"daa5d3e9e32fbd4f0c8dc7b05c18863ff3a4ee2faed268d04229c3316f659d67"} Jul 10 18:17:57.554690 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.554614 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-t89br" event={"ID":"d8158185-b1f6-45a7-af6d-458b47d5dd8e","Type":"ContainerStarted","Data":"83d1dc72571a50178b9acba720951cc162a8005c7355831ecf5bc2f8b547137c"} Jul 10 18:17:57.556018 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.555994 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" event={"ID":"c81ab235-1f9d-41c8-af87-80ee3aa5c67c","Type":"ContainerStarted","Data":"d3e55fbf796124f088d9c9705667cf71473aaa4a29437031a3568920b077ebf2"} Jul 10 18:17:57.810507 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.810321 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:17:57.893103 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:57.893055 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:17:58.567173 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:58.567137 2574 generic.go:356] "Generic (PLEG): container finished" podID="a73fabd7-58c7-4f58-883c-1fd0a6650512" containerID="120b8fcd24974cdd398f3619ad83d049748e895e8aba5234c0a0b108e65cbbcc" exitCode=0 Jul 10 18:17:58.567702 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:58.567189 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nj84h" event={"ID":"a73fabd7-58c7-4f58-883c-1fd0a6650512","Type":"ContainerDied","Data":"120b8fcd24974cdd398f3619ad83d049748e895e8aba5234c0a0b108e65cbbcc"} Jul 10 18:17:58.669175 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:58.668077 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:17:58.669175 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:58.668250 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:17:58.669175 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:58.668536 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cb47549b8-x8zsn: secret "image-registry-tls" not found Jul 10 18:17:58.669175 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:58.668616 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls podName:8b5164ee-1735-42c2-8cf1-766f022b5703 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:02.668591724 +0000 UTC m=+42.081876521 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls") pod "image-registry-7cb47549b8-x8zsn" (UID: "8b5164ee-1735-42c2-8cf1-766f022b5703") : secret "image-registry-tls" not found Jul 10 18:17:58.770085 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:58.769281 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:17:58.770085 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:58.769326 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-cqgzn\" (UID: \"ae73be3c-7b12-4e96-9afb-08b5e3f10c99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" Jul 10 18:17:58.770085 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:58.769379 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:58.770085 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:58.769416 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:17:58.770085 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:58.769563 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle podName:b8a376b5-b2dc-439a-aa9c-b1bd4f37252d nodeName:}" failed. No retries permitted until 2026-07-10 18:18:02.769548242 +0000 UTC m=+42.182833020 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle") pod "router-default-57f678b4d6-q2rhg" (UID: "b8a376b5-b2dc-439a-aa9c-b1bd4f37252d") : configmap references non-existent config key: service-ca.crt Jul 10 18:17:58.770085 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:58.769630 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:17:58.770085 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:58.769653 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls podName:20f79a38-b179-4205-bf87-71983574bfdd nodeName:}" failed. No retries permitted until 2026-07-10 18:18:02.769646165 +0000 UTC m=+42.182930930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-scpbz" (UID: "20f79a38-b179-4205-bf87-71983574bfdd") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:17:58.770085 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:58.769690 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:17:58.770085 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:58.769709 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls podName:ae73be3c-7b12-4e96-9afb-08b5e3f10c99 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:02.769702636 +0000 UTC m=+42.182987400 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls") pod "cluster-samples-operator-65db858894-cqgzn" (UID: "ae73be3c-7b12-4e96-9afb-08b5e3f10c99") : secret "samples-operator-tls" not found Jul 10 18:17:58.770085 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:58.769775 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:17:58.770085 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:58.769810 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs podName:b8a376b5-b2dc-439a-aa9c-b1bd4f37252d nodeName:}" failed. No retries permitted until 2026-07-10 18:18:02.769803393 +0000 UTC m=+42.183088172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs") pod "router-default-57f678b4d6-q2rhg" (UID: "b8a376b5-b2dc-439a-aa9c-b1bd4f37252d") : secret "router-metrics-certs-default" not found Jul 10 18:17:58.871487 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:58.870868 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:17:58.871487 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:58.871027 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:17:58.871487 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:17:58.871085 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls podName:d29817f3-db17-4d2b-9666-20e677724b75 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:02.87106603 +0000 UTC m=+42.284350801 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls") pod "dns-default-7nprf" (UID: "d29817f3-db17-4d2b-9666-20e677724b75") : secret "dns-default-metrics-tls" not found Jul 10 18:17:59.488447 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:59.488362 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:17:59.576481 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:59.576245 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nj84h" event={"ID":"a73fabd7-58c7-4f58-883c-1fd0a6650512","Type":"ContainerStarted","Data":"5ef827920b0b8ff98a3eb47b58f0be37ebf287ff22b086d09f30c4b73695ef08"} Jul 10 18:17:59.578477 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:59.578445 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vp5ls" event={"ID":"e953a150-6569-49ea-8f29-fc6503cf328a","Type":"ContainerStarted","Data":"747ea80e216e6d5ff0a3a31c1845b6723bcb1035f58e5d8573a012510d10096f"} Jul 10 18:17:59.587555 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:59.587384 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:17:59.604386 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:59.603753 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nj84h" podStartSLOduration=6.068231928 podStartE2EDuration="38.603731167s" totalImagesPullingTime="32.535499239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:59.601925596 +0000 UTC m=+39.015210384" watchObservedRunningTime="2026-07-10 18:17:59.603731167 +0000 UTC m=+39.017015955" Jul 10 18:17:59.620274 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:59.620152 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vp5ls" podStartSLOduration=22.001083123 podStartE2EDuration="23.620131249s" totalImagesPullingTime="1.619048126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:59.618249368 +0000 UTC m=+39.031534156" watchObservedRunningTime="2026-07-10 18:17:59.620131249 +0000 UTC m=+39.033416038" Jul 10 18:17:59.689801 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:59.689711 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:17:59.788023 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:59.787938 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:17:59.949933 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:17:59.949897 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:00.015638 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:00.015578 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:00.873422 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:00.873382 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:00.974630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:00.974600 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:02.135456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:02.135418 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:02.208607 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:02.208554 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:02.707504 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:02.707464 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:18:02.707805 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:02.707640 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:18:02.707805 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:02.707664 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cb47549b8-x8zsn: secret "image-registry-tls" not found Jul 10 18:18:02.707805 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:02.707724 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls podName:8b5164ee-1735-42c2-8cf1-766f022b5703 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:10.707705821 +0000 UTC m=+50.120990585 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls") pod "image-registry-7cb47549b8-x8zsn" (UID: "8b5164ee-1735-42c2-8cf1-766f022b5703") : secret "image-registry-tls" not found Jul 10 18:18:02.808272 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:02.808236 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:18:02.808467 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:02.808285 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-cqgzn\" (UID: \"ae73be3c-7b12-4e96-9afb-08b5e3f10c99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" Jul 10 18:18:02.808467 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:02.808346 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:18:02.808467 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:02.808395 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:18:02.808467 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:02.808435 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:18:02.808658 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:02.808491 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:18:02.808658 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:02.808494 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:18:02.808658 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:02.808501 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls podName:20f79a38-b179-4205-bf87-71983574bfdd nodeName:}" failed. No retries permitted until 2026-07-10 18:18:10.80848512 +0000 UTC m=+50.221769885 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-scpbz" (UID: "20f79a38-b179-4205-bf87-71983574bfdd") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:18:02.808658 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:02.808605 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle podName:b8a376b5-b2dc-439a-aa9c-b1bd4f37252d nodeName:}" failed. No retries permitted until 2026-07-10 18:18:10.808586332 +0000 UTC m=+50.221871098 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle") pod "router-default-57f678b4d6-q2rhg" (UID: "b8a376b5-b2dc-439a-aa9c-b1bd4f37252d") : configmap references non-existent config key: service-ca.crt Jul 10 18:18:02.808658 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:02.808620 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs podName:b8a376b5-b2dc-439a-aa9c-b1bd4f37252d nodeName:}" failed. No retries permitted until 2026-07-10 18:18:10.808612322 +0000 UTC m=+50.221897093 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs") pod "router-default-57f678b4d6-q2rhg" (UID: "b8a376b5-b2dc-439a-aa9c-b1bd4f37252d") : secret "router-metrics-certs-default" not found Jul 10 18:18:02.808658 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:02.808632 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls podName:ae73be3c-7b12-4e96-9afb-08b5e3f10c99 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:10.808624729 +0000 UTC m=+50.221909494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls") pod "cluster-samples-operator-65db858894-cqgzn" (UID: "ae73be3c-7b12-4e96-9afb-08b5e3f10c99") : secret "samples-operator-tls" not found Jul 10 18:18:02.909685 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:02.909642 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:18:02.909884 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:02.909818 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:18:02.909953 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:02.909902 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls podName:d29817f3-db17-4d2b-9666-20e677724b75 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:10.909878621 +0000 UTC m=+50.323163398 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls") pod "dns-default-7nprf" (UID: "d29817f3-db17-4d2b-9666-20e677724b75") : secret "dns-default-metrics-tls" not found Jul 10 18:18:03.531879 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:03.531832 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:03.632762 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:03.632708 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:05.390054 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.390024 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4n9wp"] Jul 10 18:18:05.405006 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.404962 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4n9wp"] Jul 10 18:18:05.405194 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.405080 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:05.407736 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.407711 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 18:18:05.408268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.408248 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qtcxd\"" Jul 10 18:18:05.408783 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.408616 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 18:18:05.413367 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.413045 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:05.478643 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.478614 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:05.536839 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.536798 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:05.537007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.536870 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0533d63c-4951-4049-9f06-119eda06b602-data-volume\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:05.537007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.536917 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79w49\" (UniqueName: \"kubernetes.io/projected/0533d63c-4951-4049-9f06-119eda06b602-kube-api-access-79w49\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:05.537156 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.537009 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0533d63c-4951-4049-9f06-119eda06b602-crio-socket\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:05.537156 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.537043 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:05.597239 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.597199 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cpmcx" event={"ID":"60217543-943f-4011-8a09-132f51157b28","Type":"ContainerStarted","Data":"a8d4a0e57d81909f47b65ae80f44706f5fb8792216d7547a0db1fef095f6c7d0"} Jul 10 18:18:05.598962 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.598937 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/0.log" Jul 10 18:18:05.599133 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.598981 2574 generic.go:356] "Generic (PLEG): container finished" podID="e3770e0a-c127-454b-b0dc-6e4d4a9c1628" containerID="0ed1c6b69d707edc560d416296b37bfc0fd58ac9a8462a3b6512a6986d89b59b" exitCode=255 Jul 10 18:18:05.599133 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.599055 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-cjjrj" event={"ID":"e3770e0a-c127-454b-b0dc-6e4d4a9c1628","Type":"ContainerDied","Data":"0ed1c6b69d707edc560d416296b37bfc0fd58ac9a8462a3b6512a6986d89b59b"} Jul 10 18:18:05.599304 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.599281 2574 scope.go:117] "RemoveContainer" containerID="0ed1c6b69d707edc560d416296b37bfc0fd58ac9a8462a3b6512a6986d89b59b" Jul 10 18:18:05.600924 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.600904 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-t89br" event={"ID":"d8158185-b1f6-45a7-af6d-458b47d5dd8e","Type":"ContainerStarted","Data":"49d891a6d08e4d48972c22eef37d7b009d73ac89df8a069e2014251d6e61e8de"} Jul 10 18:18:05.602569 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.602546 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" event={"ID":"c81ab235-1f9d-41c8-af87-80ee3aa5c67c","Type":"ContainerStarted","Data":"af81f93869832316e7f54595a49be660770788679af5e71a3d9fc0d6e8474cbf"} Jul 10 18:18:05.604110 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.604071 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" event={"ID":"06f82976-1aae-4012-bac5-792e61e2c8af","Type":"ContainerStarted","Data":"47b79530b4d8cbb9eba3c4ef63e29cb0e32db4f654a5068ecee01cdec918bdf9"} Jul 10 18:18:05.605516 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.605494 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" event={"ID":"d6ade376-624c-4dfc-858b-1940181dfbb3","Type":"ContainerStarted","Data":"241b9656f070a885beb75f0edd0db138e2d17d66bde0e95c0c21042dc5eea589"} Jul 10 18:18:05.613580 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.613534 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cpmcx" podStartSLOduration=4.240763884 podStartE2EDuration="11.613516579s" totalImagesPullingTime="7.372752695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:05.612118907 +0000 UTC m=+45.025403689" watchObservedRunningTime="2026-07-10 18:18:05.613516579 +0000 UTC m=+45.026801369" Jul 10 18:18:05.631107 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.631040 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" podStartSLOduration=22.228897995 podStartE2EDuration="29.631018889s" totalImagesPullingTime="7.402120894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:05.629814328 +0000 UTC m=+45.043099116" watchObservedRunningTime="2026-07-10 18:18:05.631018889 +0000 UTC m=+45.044303677" Jul 10 18:18:05.638090 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.638055 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0533d63c-4951-4049-9f06-119eda06b602-data-volume\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:05.638218 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.638089 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79w49\" (UniqueName: \"kubernetes.io/projected/0533d63c-4951-4049-9f06-119eda06b602-kube-api-access-79w49\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:05.638218 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.638149 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0533d63c-4951-4049-9f06-119eda06b602-crio-socket\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:05.638218 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.638172 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:05.638375 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.638251 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:05.638375 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:05.638326 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:18:05.638375 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:05.638369 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm podName:0533d63c-4951-4049-9f06-119eda06b602 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:06.13835711 +0000 UTC m=+45.551641876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4n9wp" (UID: "0533d63c-4951-4049-9f06-119eda06b602") : configmap "kube-rbac-proxy" not found Jul 10 18:18:05.638529 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.638386 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0533d63c-4951-4049-9f06-119eda06b602-data-volume\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:05.638529 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:05.638494 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:18:05.638625 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:05.638584 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls podName:0533d63c-4951-4049-9f06-119eda06b602 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:06.138569377 +0000 UTC m=+45.551854147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4n9wp" (UID: "0533d63c-4951-4049-9f06-119eda06b602") : secret "insights-runtime-extractor-tls" not found Jul 10 18:18:05.638776 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.638742 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0533d63c-4951-4049-9f06-119eda06b602-crio-socket\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:05.652397 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.652369 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79w49\" (UniqueName: \"kubernetes.io/projected/0533d63c-4951-4049-9f06-119eda06b602-kube-api-access-79w49\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:05.674797 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.674739 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-t89br" podStartSLOduration=22.300716494 podStartE2EDuration="29.674720083s" totalImagesPullingTime="7.374003589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:05.674488293 +0000 UTC m=+45.087773079" watchObservedRunningTime="2026-07-10 18:18:05.674720083 +0000 UTC m=+45.088004855" Jul 10 18:18:05.694936 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.694406 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" podStartSLOduration=22.32087427 podStartE2EDuration="29.694385307s" totalImagesPullingTime="7.373511037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:05.693370564 +0000 UTC m=+45.106655354" watchObservedRunningTime="2026-07-10 18:18:05.694385307 +0000 UTC m=+45.107670092" Jul 10 18:18:05.715827 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:05.715768 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" podStartSLOduration=22.341767163 podStartE2EDuration="29.715751706s" totalImagesPullingTime="7.373984543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:05.714576713 +0000 UTC m=+45.127861523" watchObservedRunningTime="2026-07-10 18:18:05.715751706 +0000 UTC m=+45.129036492" Jul 10 18:18:06.143892 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:06.143841 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:06.144091 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:06.143979 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:06.144091 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:06.143998 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:18:06.144091 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:06.144064 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls podName:0533d63c-4951-4049-9f06-119eda06b602 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:07.144046859 +0000 UTC m=+46.557331624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4n9wp" (UID: "0533d63c-4951-4049-9f06-119eda06b602") : secret "insights-runtime-extractor-tls" not found Jul 10 18:18:06.144278 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:06.144123 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:18:06.144278 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:06.144182 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm podName:0533d63c-4951-4049-9f06-119eda06b602 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:07.14416576 +0000 UTC m=+46.557450529 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4n9wp" (UID: "0533d63c-4951-4049-9f06-119eda06b602") : configmap "kube-rbac-proxy" not found Jul 10 18:18:06.387505 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:06.387470 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:06.484304 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:06.484266 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:06.610132 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:06.610081 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/1.log" Jul 10 18:18:06.610494 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:06.610479 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/0.log" Jul 10 18:18:06.610546 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:06.610517 2574 generic.go:356] "Generic (PLEG): container finished" podID="e3770e0a-c127-454b-b0dc-6e4d4a9c1628" containerID="380aefc8513fc138093a6390cb4bc684747270056d3d8c7b4996a6d33dd02810" exitCode=255 Jul 10 18:18:06.610659 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:06.610629 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-cjjrj" event={"ID":"e3770e0a-c127-454b-b0dc-6e4d4a9c1628","Type":"ContainerDied","Data":"380aefc8513fc138093a6390cb4bc684747270056d3d8c7b4996a6d33dd02810"} Jul 10 18:18:06.610755 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:06.610739 2574 scope.go:117] "RemoveContainer" containerID="0ed1c6b69d707edc560d416296b37bfc0fd58ac9a8462a3b6512a6986d89b59b" Jul 10 18:18:06.610846 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:06.610831 2574 scope.go:117] "RemoveContainer" containerID="380aefc8513fc138093a6390cb4bc684747270056d3d8c7b4996a6d33dd02810" Jul 10 18:18:06.611054 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:06.611030 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-cjjrj_openshift-console-operator(e3770e0a-c127-454b-b0dc-6e4d4a9c1628)\"" pod="openshift-console-operator/console-operator-7866497494-cjjrj" podUID="e3770e0a-c127-454b-b0dc-6e4d4a9c1628" Jul 10 18:18:06.906792 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:06.906716 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:07.000628 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:07.000594 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:07.106705 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:07.106672 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:07.154857 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:07.154816 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:07.155056 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:07.154935 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:07.155056 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:07.154964 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:18:07.155056 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:07.155042 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls podName:0533d63c-4951-4049-9f06-119eda06b602 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:09.155026075 +0000 UTC m=+48.568310840 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4n9wp" (UID: "0533d63c-4951-4049-9f06-119eda06b602") : secret "insights-runtime-extractor-tls" not found Jul 10 18:18:07.155056 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:07.155045 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:18:07.155247 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:07.155123 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm podName:0533d63c-4951-4049-9f06-119eda06b602 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:09.155091274 +0000 UTC m=+48.568376039 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4n9wp" (UID: "0533d63c-4951-4049-9f06-119eda06b602") : configmap "kube-rbac-proxy" not found Jul 10 18:18:07.157302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:07.157249 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7qhmx_42571c86-4eaf-44a2-868e-1e023a349549/dns-node-resolver/0.log" Jul 10 18:18:07.204878 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:07.204849 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:07.615605 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:07.615578 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/1.log" Jul 10 18:18:07.615980 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:07.615910 2574 scope.go:117] "RemoveContainer" containerID="380aefc8513fc138093a6390cb4bc684747270056d3d8c7b4996a6d33dd02810" Jul 10 18:18:07.616135 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:07.616112 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-cjjrj_openshift-console-operator(e3770e0a-c127-454b-b0dc-6e4d4a9c1628)\"" pod="openshift-console-operator/console-operator-7866497494-cjjrj" podUID="e3770e0a-c127-454b-b0dc-6e4d4a9c1628" Jul 10 18:18:07.941249 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:07.941214 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:07.999656 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:07.999623 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:08.358427 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:08.358354 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c8b4p_1e02735c-c7f7-4070-9a1f-0410bace9cd9/node-ca/0.log" Jul 10 18:18:08.733274 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:08.733242 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:08.788069 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:08.788032 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:08.957372 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:08.957345 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cpmcx_60217543-943f-4011-8a09-132f51157b28/serve-healthcheck-canary/0.log" Jul 10 18:18:09.172957 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:09.172877 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:09.173118 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:09.173000 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:09.173172 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:09.173003 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:18:09.173206 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:09.173191 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm podName:0533d63c-4951-4049-9f06-119eda06b602 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:13.173172545 +0000 UTC m=+52.586457309 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4n9wp" (UID: "0533d63c-4951-4049-9f06-119eda06b602") : configmap "kube-rbac-proxy" not found Jul 10 18:18:09.173249 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:09.173077 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:18:09.173282 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:09.173263 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls podName:0533d63c-4951-4049-9f06-119eda06b602 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:13.173249482 +0000 UTC m=+52.586534254 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4n9wp" (UID: "0533d63c-4951-4049-9f06-119eda06b602") : secret "insights-runtime-extractor-tls" not found Jul 10 18:18:10.279420 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:10.279390 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:10.789722 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:10.789679 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:18:10.789942 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:10.789865 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 18:18:10.789942 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:10.789889 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cb47549b8-x8zsn: secret "image-registry-tls" not found Jul 10 18:18:10.790064 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:10.789961 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls podName:8b5164ee-1735-42c2-8cf1-766f022b5703 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:26.789938428 +0000 UTC m=+66.203223193 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls") pod "image-registry-7cb47549b8-x8zsn" (UID: "8b5164ee-1735-42c2-8cf1-766f022b5703") : secret "image-registry-tls" not found Jul 10 18:18:10.890312 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:10.890269 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:18:10.890480 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:10.890342 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:18:10.890480 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:10.890431 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 18:18:10.890480 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:10.890451 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:18:10.890480 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:10.890462 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle podName:b8a376b5-b2dc-439a-aa9c-b1bd4f37252d nodeName:}" failed. No retries permitted until 2026-07-10 18:18:26.890445067 +0000 UTC m=+66.303729861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle") pod "router-default-57f678b4d6-q2rhg" (UID: "b8a376b5-b2dc-439a-aa9c-b1bd4f37252d") : configmap references non-existent config key: service-ca.crt Jul 10 18:18:10.890614 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:10.890493 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs podName:b8a376b5-b2dc-439a-aa9c-b1bd4f37252d nodeName:}" failed. No retries permitted until 2026-07-10 18:18:26.890485997 +0000 UTC m=+66.303770761 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs") pod "router-default-57f678b4d6-q2rhg" (UID: "b8a376b5-b2dc-439a-aa9c-b1bd4f37252d") : secret "router-metrics-certs-default" not found Jul 10 18:18:10.890614 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:10.890514 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 18:18:10.890614 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:10.890522 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-cqgzn\" (UID: \"ae73be3c-7b12-4e96-9afb-08b5e3f10c99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" Jul 10 18:18:10.890614 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:10.890560 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls podName:20f79a38-b179-4205-bf87-71983574bfdd nodeName:}" failed. No retries permitted until 2026-07-10 18:18:26.890546259 +0000 UTC m=+66.303831027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-scpbz" (UID: "20f79a38-b179-4205-bf87-71983574bfdd") : secret "cluster-monitoring-operator-tls" not found Jul 10 18:18:10.890740 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:10.890615 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 18:18:10.890740 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:10.890671 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls podName:ae73be3c-7b12-4e96-9afb-08b5e3f10c99 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:26.89066152 +0000 UTC m=+66.303946290 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls") pod "cluster-samples-operator-65db858894-cqgzn" (UID: "ae73be3c-7b12-4e96-9afb-08b5e3f10c99") : secret "samples-operator-tls" not found Jul 10 18:18:10.991703 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:10.991664 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:18:10.991906 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:10.991823 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:18:10.991906 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:10.991890 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls podName:d29817f3-db17-4d2b-9666-20e677724b75 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:26.991872707 +0000 UTC m=+66.405157485 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls") pod "dns-default-7nprf" (UID: "d29817f3-db17-4d2b-9666-20e677724b75") : secret "dns-default-metrics-tls" not found Jul 10 18:18:11.220334 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:11.220296 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:12.713416 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:12.713381 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:12.785041 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:12.785005 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:13.211387 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:13.211354 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:13.211562 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:13.211452 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:13.211624 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:13.211602 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:18:13.211680 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:13.211670 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls podName:0533d63c-4951-4049-9f06-119eda06b602 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:21.211655711 +0000 UTC m=+60.624940478 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4n9wp" (UID: "0533d63c-4951-4049-9f06-119eda06b602") : secret "insights-runtime-extractor-tls" not found Jul 10 18:18:13.211914 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:13.211895 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0533d63c-4951-4049-9f06-119eda06b602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:13.400274 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:13.400240 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:13.521076 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:13.520992 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:13.624407 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:13.624367 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:13.725184 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:13.725151 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:13.836255 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:13.836163 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:13.923990 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:13.923954 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:14.294462 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:14.294430 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:14.689830 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:14.689800 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:15.304609 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:15.304577 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:15.348120 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:15.348063 2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:18:15.348290 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:15.348146 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:18:15.348530 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:15.348515 2574 scope.go:117] "RemoveContainer" containerID="380aefc8513fc138093a6390cb4bc684747270056d3d8c7b4996a6d33dd02810" Jul 10 18:18:15.348681 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:15.348661 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-cjjrj_openshift-console-operator(e3770e0a-c127-454b-b0dc-6e4d4a9c1628)\"" pod="openshift-console-operator/console-operator-7866497494-cjjrj" podUID="e3770e0a-c127-454b-b0dc-6e4d4a9c1628" Jul 10 18:18:15.697342 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:15.697310 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:15.894045 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:15.893995 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:16.092328 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:16.092253 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:16.289203 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:16.289170 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:16.892065 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:16.892032 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:17.091558 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:17.091528 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:17.214640 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:17.214612 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:17.492747 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:17.492680 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:17.690547 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:17.690510 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:18.090923 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:18.090889 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:18.295811 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:18.295766 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:18.702659 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:18.702623 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:19.298338 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:19.298308 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:19.696657 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:19.696624 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:19.795503 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:19.795470 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:19.901524 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:19.901491 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:20.491980 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:20.491947 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:20.899078 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:20.898996 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:21.097204 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:21.097171 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:21.282532 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:21.282486 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:21.284857 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:21.284828 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0533d63c-4951-4049-9f06-119eda06b602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4n9wp\" (UID: \"0533d63c-4951-4049-9f06-119eda06b602\") " pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:21.318719 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:21.318685 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qtcxd\"" Jul 10 18:18:21.327494 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:21.327473 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4n9wp" Jul 10 18:18:21.490184 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:21.490153 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4n9wp"] Jul 10 18:18:21.493540 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:18:21.493514 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0533d63c_4951_4049_9f06_119eda06b602.slice/crio-29dbbfd8d205b6e10e10a76fc4c7b6fdd179f8ceaabd821435917ad05ceeca92 WatchSource:0}: Error finding container 29dbbfd8d205b6e10e10a76fc4c7b6fdd179f8ceaabd821435917ad05ceeca92: Status 404 returned error can't find the container with id 29dbbfd8d205b6e10e10a76fc4c7b6fdd179f8ceaabd821435917ad05ceeca92 Jul 10 18:18:21.507077 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:21.507050 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:21.538209 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:21.538123 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9c9bx" Jul 10 18:18:21.657026 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:21.656994 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4n9wp" event={"ID":"0533d63c-4951-4049-9f06-119eda06b602","Type":"ContainerStarted","Data":"29dbbfd8d205b6e10e10a76fc4c7b6fdd179f8ceaabd821435917ad05ceeca92"} Jul 10 18:18:22.294042 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:22.293993 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:22.496069 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:22.496029 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:22.694432 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:22.694386 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:23.103725 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:23.103593 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:23.490181 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:23.490141 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:23.894838 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:23.894749 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:24.097242 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:24.097208 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:25.673705 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:25.673666 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4n9wp" event={"ID":"0533d63c-4951-4049-9f06-119eda06b602","Type":"ContainerStarted","Data":"c74ee1659d57f076b4fe51e6f278660ad847240be5781e8c10b12cacab112d3c"} Jul 10 18:18:25.929486 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:25.929391 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:18:25.929486 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:25.929438 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs\") pod \"network-metrics-daemon-j7bj7\" (UID: \"cfcce616-ce21-4462-865f-116401b2c78f\") " pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:18:25.931981 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:25.931949 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 18:18:25.931981 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:25.931953 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 18:18:25.942772 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:25.942745 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/97b02d51-cfd3-4d11-a108-c54bf9dca12e-original-pull-secret\") pod \"global-pull-secret-syncer-84n7b\" (UID: \"97b02d51-cfd3-4d11-a108-c54bf9dca12e\") " pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:18:25.942878 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:25.942760 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfcce616-ce21-4462-865f-116401b2c78f-metrics-certs\") pod \"network-metrics-daemon-j7bj7\" (UID: \"cfcce616-ce21-4462-865f-116401b2c78f\") " pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:18:26.028494 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.028442 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84n7b" Jul 10 18:18:26.030383 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.030357 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsjf5\" (UniqueName: \"kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5\") pod \"network-check-target-56zsp\" (UID: \"c453daa4-8e53-47b8-a056-7a7c32164c45\") " pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:18:26.032789 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.032762 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsjf5\" (UniqueName: \"kubernetes.io/projected/c453daa4-8e53-47b8-a056-7a7c32164c45-kube-api-access-tsjf5\") pod \"network-check-target-56zsp\" (UID: \"c453daa4-8e53-47b8-a056-7a7c32164c45\") " pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:18:26.035266 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.035241 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xs98z\"" Jul 10 18:18:26.043424 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.043225 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j7bj7" Jul 10 18:18:26.163918 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.163888 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-84n7b"] Jul 10 18:18:26.167057 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:18:26.167029 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97b02d51_cfd3_4d11_a108_c54bf9dca12e.slice/crio-fbfc7209d4f4b07d35874e7d091b105654c206b80b8836dee1869b66b2172805 WatchSource:0}: Error finding container fbfc7209d4f4b07d35874e7d091b105654c206b80b8836dee1869b66b2172805: Status 404 returned error can't find the container with id fbfc7209d4f4b07d35874e7d091b105654c206b80b8836dee1869b66b2172805 Jul 10 18:18:26.180821 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.180770 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j7bj7"] Jul 10 18:18:26.183341 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:18:26.183319 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfcce616_ce21_4462_865f_116401b2c78f.slice/crio-268b1fbb3c4f2a7e7cd080d7cc1c1fc95b90413aad50357f71b262ce486c26f4 WatchSource:0}: Error finding container 268b1fbb3c4f2a7e7cd080d7cc1c1fc95b90413aad50357f71b262ce486c26f4: Status 404 returned error can't find the container with id 268b1fbb3c4f2a7e7cd080d7cc1c1fc95b90413aad50357f71b262ce486c26f4 Jul 10 18:18:26.323469 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.323441 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nsh8q\"" Jul 10 18:18:26.331194 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.331158 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:18:26.479251 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.479165 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-56zsp"] Jul 10 18:18:26.482738 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:18:26.482710 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc453daa4_8e53_47b8_a056_7a7c32164c45.slice/crio-dadacc92b8ec572f4712584fc6e329e287877548f3af3a7ca02f7eb933a1dd50 WatchSource:0}: Error finding container dadacc92b8ec572f4712584fc6e329e287877548f3af3a7ca02f7eb933a1dd50: Status 404 returned error can't find the container with id dadacc92b8ec572f4712584fc6e329e287877548f3af3a7ca02f7eb933a1dd50 Jul 10 18:18:26.679061 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.679021 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-84n7b" event={"ID":"97b02d51-cfd3-4d11-a108-c54bf9dca12e","Type":"ContainerStarted","Data":"fbfc7209d4f4b07d35874e7d091b105654c206b80b8836dee1869b66b2172805"} Jul 10 18:18:26.680466 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.680408 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j7bj7" event={"ID":"cfcce616-ce21-4462-865f-116401b2c78f","Type":"ContainerStarted","Data":"268b1fbb3c4f2a7e7cd080d7cc1c1fc95b90413aad50357f71b262ce486c26f4"} Jul 10 18:18:26.682780 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.682718 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4n9wp" event={"ID":"0533d63c-4951-4049-9f06-119eda06b602","Type":"ContainerStarted","Data":"5f3fc67c3dc33307a64b57c35a5c288fede219d3bcabdce4e3e210b8d3a40175"} Jul 10 18:18:26.684664 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.684625 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-56zsp" event={"ID":"c453daa4-8e53-47b8-a056-7a7c32164c45","Type":"ContainerStarted","Data":"942d226957d82279dc397ab81d84f1d63bbcd74e5c9e59c63e2a6753deb905b2"} Jul 10 18:18:26.684780 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.684669 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-56zsp" event={"ID":"c453daa4-8e53-47b8-a056-7a7c32164c45","Type":"ContainerStarted","Data":"dadacc92b8ec572f4712584fc6e329e287877548f3af3a7ca02f7eb933a1dd50"} Jul 10 18:18:26.684853 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.684778 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:18:26.699584 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.699532 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-56zsp" podStartSLOduration=65.699513361 podStartE2EDuration="1m5.699513361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:26.699252967 +0000 UTC m=+66.112537755" watchObservedRunningTime="2026-07-10 18:18:26.699513361 +0000 UTC m=+66.112798150" Jul 10 18:18:26.839215 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.838986 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:18:26.842225 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.842194 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls\") pod \"image-registry-7cb47549b8-x8zsn\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:18:26.939669 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.939584 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:18:26.939669 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.939651 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:18:26.939918 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.939717 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:18:26.939918 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.939756 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-cqgzn\" (UID: \"ae73be3c-7b12-4e96-9afb-08b5e3f10c99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" Jul 10 18:18:26.940748 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.940713 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-service-ca-bundle\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:18:26.942755 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.942729 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/20f79a38-b179-4205-bf87-71983574bfdd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-scpbz\" (UID: \"20f79a38-b179-4205-bf87-71983574bfdd\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:18:26.942871 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.942778 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae73be3c-7b12-4e96-9afb-08b5e3f10c99-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-cqgzn\" (UID: \"ae73be3c-7b12-4e96-9afb-08b5e3f10c99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" Jul 10 18:18:26.943015 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:26.942993 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8a376b5-b2dc-439a-aa9c-b1bd4f37252d-metrics-certs\") pod \"router-default-57f678b4d6-q2rhg\" (UID: \"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d\") " pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:18:27.040897 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.040857 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:18:27.044059 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.044013 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d29817f3-db17-4d2b-9666-20e677724b75-metrics-tls\") pod \"dns-default-7nprf\" (UID: \"d29817f3-db17-4d2b-9666-20e677724b75\") " pod="openshift-dns/dns-default-7nprf" Jul 10 18:18:27.064723 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.064476 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6v2rr\"" Jul 10 18:18:27.072445 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.072394 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:18:27.112946 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.112912 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4sqzg\"" Jul 10 18:18:27.120848 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.120807 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" Jul 10 18:18:27.135565 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.135492 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vm4x9\"" Jul 10 18:18:27.143913 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.143891 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:18:27.168315 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.168286 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-r2674\"" Jul 10 18:18:27.176443 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.176363 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" Jul 10 18:18:27.261345 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.261310 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p5qqq\"" Jul 10 18:18:27.269465 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.269434 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7nprf" Jul 10 18:18:27.348847 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.348750 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cb47549b8-x8zsn"] Jul 10 18:18:27.359067 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:18:27.358990 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b5164ee_1735_42c2_8cf1_766f022b5703.slice/crio-5032fbe07ac47d05895229e5b488d796fd2bc3da42a15ac40c3bae22d5aaa19e WatchSource:0}: Error finding container 5032fbe07ac47d05895229e5b488d796fd2bc3da42a15ac40c3bae22d5aaa19e: Status 404 returned error can't find the container with id 5032fbe07ac47d05895229e5b488d796fd2bc3da42a15ac40c3bae22d5aaa19e Jul 10 18:18:27.370673 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.370428 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz"] Jul 10 18:18:27.389346 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.389262 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57f678b4d6-q2rhg"] Jul 10 18:18:27.410911 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.410821 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn"] Jul 10 18:18:27.478046 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.478011 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7nprf"] Jul 10 18:18:27.564362 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:18:27.564324 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f79a38_b179_4205_bf87_71983574bfdd.slice/crio-3e8175e2210d1b33af33c1eff1b5510eb75bdd9077f5b5552d96a5ab6e39d00d WatchSource:0}: Error finding container 3e8175e2210d1b33af33c1eff1b5510eb75bdd9077f5b5552d96a5ab6e39d00d: Status 404 returned error can't find the container with id 3e8175e2210d1b33af33c1eff1b5510eb75bdd9077f5b5552d96a5ab6e39d00d Jul 10 18:18:27.565343 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:18:27.565225 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a376b5_b2dc_439a_aa9c_b1bd4f37252d.slice/crio-d11d42f60bc9795b866a716d7830b8080fc127122435ef293cf059ec657751e8 WatchSource:0}: Error finding container d11d42f60bc9795b866a716d7830b8080fc127122435ef293cf059ec657751e8: Status 404 returned error can't find the container with id d11d42f60bc9795b866a716d7830b8080fc127122435ef293cf059ec657751e8 Jul 10 18:18:27.566956 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:18:27.566928 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29817f3_db17_4d2b_9666_20e677724b75.slice/crio-505c7f98693f3bb9d84994ceea4c7571b01d820d9b5dc062b86f160a466c6ffa WatchSource:0}: Error finding container 505c7f98693f3bb9d84994ceea4c7571b01d820d9b5dc062b86f160a466c6ffa: Status 404 returned error can't find the container with id 505c7f98693f3bb9d84994ceea4c7571b01d820d9b5dc062b86f160a466c6ffa Jul 10 18:18:27.690552 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.690439 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" event={"ID":"ae73be3c-7b12-4e96-9afb-08b5e3f10c99","Type":"ContainerStarted","Data":"23e7990e46ec24ab54ce7782a2c6987000eca2d712f472c1919349ef98412ca0"} Jul 10 18:18:27.691642 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.691608 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7nprf" event={"ID":"d29817f3-db17-4d2b-9666-20e677724b75","Type":"ContainerStarted","Data":"505c7f98693f3bb9d84994ceea4c7571b01d820d9b5dc062b86f160a466c6ffa"} Jul 10 18:18:27.692929 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.692887 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57f678b4d6-q2rhg" event={"ID":"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d","Type":"ContainerStarted","Data":"d11d42f60bc9795b866a716d7830b8080fc127122435ef293cf059ec657751e8"} Jul 10 18:18:27.694533 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.694498 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" event={"ID":"8b5164ee-1735-42c2-8cf1-766f022b5703","Type":"ContainerStarted","Data":"a9addd08ac174f05fca700f60a806457b11701eebb919995f427465b5b9f9079"} Jul 10 18:18:27.694655 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.694539 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" event={"ID":"8b5164ee-1735-42c2-8cf1-766f022b5703","Type":"ContainerStarted","Data":"5032fbe07ac47d05895229e5b488d796fd2bc3da42a15ac40c3bae22d5aaa19e"} Jul 10 18:18:27.694655 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.694613 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:18:27.695830 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.695804 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" event={"ID":"20f79a38-b179-4205-bf87-71983574bfdd","Type":"ContainerStarted","Data":"3e8175e2210d1b33af33c1eff1b5510eb75bdd9077f5b5552d96a5ab6e39d00d"} Jul 10 18:18:27.713687 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:27.713632 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" podStartSLOduration=66.713618972 podStartE2EDuration="1m6.713618972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:27.712641312 +0000 UTC m=+67.125926100" watchObservedRunningTime="2026-07-10 18:18:27.713618972 +0000 UTC m=+67.126903737" Jul 10 18:18:29.310476 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.310397 2574 scope.go:117] "RemoveContainer" containerID="380aefc8513fc138093a6390cb4bc684747270056d3d8c7b4996a6d33dd02810" Jul 10 18:18:29.564811 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.564690 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:29.662218 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.662167 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:29.709135 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.708500 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57f678b4d6-q2rhg" event={"ID":"b8a376b5-b2dc-439a-aa9c-b1bd4f37252d","Type":"ContainerStarted","Data":"00c3429775c2560fd887bcb84eab2fcffcc147dcb28f19a3b0a806b55ca7e6e7"} Jul 10 18:18:29.713027 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.712538 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j7bj7" event={"ID":"cfcce616-ce21-4462-865f-116401b2c78f","Type":"ContainerStarted","Data":"f59b8c7075f2326a961f42a60d98e7b00293600bbac98796d90d66ee43ebc84c"} Jul 10 18:18:29.713027 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.712598 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j7bj7" event={"ID":"cfcce616-ce21-4462-865f-116401b2c78f","Type":"ContainerStarted","Data":"065296b49cd67e5783e83a03d3be706abd3d9a340f6a9481667068ed396801a1"} Jul 10 18:18:29.717712 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.717572 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4n9wp" event={"ID":"0533d63c-4951-4049-9f06-119eda06b602","Type":"ContainerStarted","Data":"2e736d4b806c1b2817086beefbf5a946ff9f728997a2923c931625e5d51f4128"} Jul 10 18:18:29.720182 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.720070 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:18:29.721782 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.720732 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/1.log" Jul 10 18:18:29.721782 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.720767 2574 generic.go:356] "Generic (PLEG): container finished" podID="e3770e0a-c127-454b-b0dc-6e4d4a9c1628" containerID="1f74a64d77d94357b354d3622396f0ff40edfb7eebd77aa146453c62ec82ba6b" exitCode=255 Jul 10 18:18:29.721782 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.720807 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-cjjrj" event={"ID":"e3770e0a-c127-454b-b0dc-6e4d4a9c1628","Type":"ContainerDied","Data":"1f74a64d77d94357b354d3622396f0ff40edfb7eebd77aa146453c62ec82ba6b"} Jul 10 18:18:29.721782 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.720849 2574 scope.go:117] "RemoveContainer" containerID="380aefc8513fc138093a6390cb4bc684747270056d3d8c7b4996a6d33dd02810" Jul 10 18:18:29.721782 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.721264 2574 scope.go:117] "RemoveContainer" containerID="1f74a64d77d94357b354d3622396f0ff40edfb7eebd77aa146453c62ec82ba6b" Jul 10 18:18:29.721782 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:29.721433 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-cjjrj_openshift-console-operator(e3770e0a-c127-454b-b0dc-6e4d4a9c1628)\"" pod="openshift-console-operator/console-operator-7866497494-cjjrj" podUID="e3770e0a-c127-454b-b0dc-6e4d4a9c1628" Jul 10 18:18:29.740135 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.739681 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-57f678b4d6-q2rhg" podStartSLOduration=53.739658925 podStartE2EDuration="53.739658925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:29.738058363 +0000 UTC m=+69.151343153" watchObservedRunningTime="2026-07-10 18:18:29.739658925 +0000 UTC m=+69.152943715" Jul 10 18:18:29.764792 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.764459 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-j7bj7" podStartSLOduration=67.356866015 podStartE2EDuration="1m8.764438552s" totalImagesPullingTime="1.407572537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:29.763810074 +0000 UTC m=+69.177094862" watchObservedRunningTime="2026-07-10 18:18:29.764438552 +0000 UTC m=+69.177723342" Jul 10 18:18:29.769640 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.769580 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:29.813163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.812181 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4n9wp" podStartSLOduration=17.293194241 podStartE2EDuration="24.812158679s" totalImagesPullingTime="7.518964438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:18:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:29.811336814 +0000 UTC m=+69.224621602" watchObservedRunningTime="2026-07-10 18:18:29.812158679 +0000 UTC m=+69.225443467" Jul 10 18:18:29.855402 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.855317 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:29.952394 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:29.952321 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:30.144870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:30.144785 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:18:30.147981 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:30.147936 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:18:30.585375 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:30.585335 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:30.727277 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:30.727238 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:18:30.728134 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:30.728081 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:18:30.729149 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:30.729129 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-57f678b4d6-q2rhg" Jul 10 18:18:30.993710 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:30.993681 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:31.094233 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:31.094194 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:32.273672 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:32.273637 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:32.374239 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:32.374205 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:32.431562 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:32.431533 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:33.331068 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.331033 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:33.474607 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.474562 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:33.605880 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.605785 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:33.741119 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.741052 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" event={"ID":"20f79a38-b179-4205-bf87-71983574bfdd","Type":"ContainerStarted","Data":"a1349ec14140e1c0247db18b17ca38c5ae1878a2efb67512500caded1efbb303"} Jul 10 18:18:33.742653 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.742624 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-84n7b" event={"ID":"97b02d51-cfd3-4d11-a108-c54bf9dca12e","Type":"ContainerStarted","Data":"3a5dbe2838cef99cae66e449f2d174dd923a0acb78ff0bbf8dcab571f1b4630b"} Jul 10 18:18:33.744290 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.744267 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" event={"ID":"ae73be3c-7b12-4e96-9afb-08b5e3f10c99","Type":"ContainerStarted","Data":"e1857b36a46411021a0564e04799a1c868fa994d8aa87181f7c5432d20e84287"} Jul 10 18:18:33.744390 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.744323 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" event={"ID":"ae73be3c-7b12-4e96-9afb-08b5e3f10c99","Type":"ContainerStarted","Data":"e1bd6c098c426409774423e57d15ae6cd7145ba8d0512c83dd5829e2025eb693"} Jul 10 18:18:33.746116 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.746074 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7nprf" event={"ID":"d29817f3-db17-4d2b-9666-20e677724b75","Type":"ContainerStarted","Data":"c5ef31216802783e53d1eb5479fb8c42521f20079aabccfcc86f53b944971d3e"} Jul 10 18:18:33.746236 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.746131 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7nprf" event={"ID":"d29817f3-db17-4d2b-9666-20e677724b75","Type":"ContainerStarted","Data":"38733e17aed67238cc3e822046dcce00b1ab5684a9882efed11026008c8e3843"} Jul 10 18:18:33.746236 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.746217 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7nprf" Jul 10 18:18:33.759071 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.759022 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-scpbz" podStartSLOduration=52.422515659 podStartE2EDuration="57.759007244s" totalImagesPullingTime="5.336491585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:33.757453545 +0000 UTC m=+73.170738334" watchObservedRunningTime="2026-07-10 18:18:33.759007244 +0000 UTC m=+73.172292031" Jul 10 18:18:33.772998 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.772945 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-84n7b" podStartSLOduration=66.022580475 podStartE2EDuration="1m12.77292893s" totalImagesPullingTime="6.750348455s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:33.771294846 +0000 UTC m=+73.184579646" watchObservedRunningTime="2026-07-10 18:18:33.77292893 +0000 UTC m=+73.186213717" Jul 10 18:18:33.788497 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.788447 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-cqgzn" podStartSLOduration=52.501982974 podStartE2EDuration="57.788430372s" totalImagesPullingTime="5.286447398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:33.787856811 +0000 UTC m=+73.201141599" watchObservedRunningTime="2026-07-10 18:18:33.788430372 +0000 UTC m=+73.201715159" Jul 10 18:18:33.807155 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.807068 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7nprf" podStartSLOduration=34.474820422 podStartE2EDuration="39.807046109s" totalImagesPullingTime="5.332225687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:33.80676197 +0000 UTC m=+73.220046758" watchObservedRunningTime="2026-07-10 18:18:33.807046109 +0000 UTC m=+73.220330897" Jul 10 18:18:33.955931 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:33.955886 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:35.348161 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:35.348120 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:18:35.348620 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:35.348184 2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:18:35.348620 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:35.348494 2574 scope.go:117] "RemoveContainer" containerID="1f74a64d77d94357b354d3622396f0ff40edfb7eebd77aa146453c62ec82ba6b" Jul 10 18:18:35.348692 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:35.348651 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-cjjrj_openshift-console-operator(e3770e0a-c127-454b-b0dc-6e4d4a9c1628)\"" pod="openshift-console-operator/console-operator-7866497494-cjjrj" podUID="e3770e0a-c127-454b-b0dc-6e4d4a9c1628" Jul 10 18:18:35.609686 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:35.609584 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:35.672239 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:35.672202 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:37.643326 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:37.643290 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:37.737042 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:37.737005 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:37.931573 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:37.931528 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:40.310992 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:40.310959 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:41.819068 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:41.819035 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:41.928330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:41.928301 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:42.031172 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.031138 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:42.057845 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.057811 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-lxj76"] Jul 10 18:18:42.060175 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.060158 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.062856 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.062834 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-9lw95\"" Jul 10 18:18:42.063946 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.063914 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 10 18:18:42.063946 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.063926 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 18:18:42.063946 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.063941 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 10 18:18:42.064185 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.063930 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 10 18:18:42.076265 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.076211 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-lxj76"] Jul 10 18:18:42.077474 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.077450 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sh4hv"] Jul 10 18:18:42.079715 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.079698 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.082446 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.082422 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 18:18:42.082584 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.082563 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 18:18:42.083250 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.083233 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zjc65\"" Jul 10 18:18:42.083487 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.083470 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 18:18:42.125574 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.125529 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:42.170672 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.170636 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-textfile\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.170672 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.170674 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.170865 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.170696 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.170865 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.170784 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.170865 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.170823 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-wtmp\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.170865 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.170853 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-metrics-client-ca\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.170988 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.170888 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-accelerators-collector-config\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.170988 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.170904 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f9r2\" (UniqueName: \"kubernetes.io/projected/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-kube-api-access-5f9r2\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.170988 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.170938 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.170988 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.170967 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e821829-aa45-49a3-a7ef-676c90c57ac2-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.171137 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.170999 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8e821829-aa45-49a3-a7ef-676c90c57ac2-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.171137 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.171017 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnbcg\" (UniqueName: \"kubernetes.io/projected/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-api-access-lnbcg\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.171137 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.171051 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-sys\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.171137 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.171070 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-root\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.171137 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.171085 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-tls\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.271885 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.271755 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8e821829-aa45-49a3-a7ef-676c90c57ac2-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.271885 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.271790 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnbcg\" (UniqueName: \"kubernetes.io/projected/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-api-access-lnbcg\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.271885 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.271818 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-sys\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.271885 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.271843 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-root\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.271885 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.271887 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-root\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.272228 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.271885 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-tls\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.272228 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.271929 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-textfile\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.272228 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.271953 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.272228 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.271960 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-sys\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.272228 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.271972 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.272462 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.272018 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.272527 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.272501 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-wtmp\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.272646 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.272617 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8e821829-aa45-49a3-a7ef-676c90c57ac2-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.272764 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.272568 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-metrics-client-ca\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.272825 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.272775 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-accelerators-collector-config\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.272825 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.272817 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f9r2\" (UniqueName: \"kubernetes.io/projected/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-kube-api-access-5f9r2\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.272924 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.272830 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-wtmp\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.272924 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.272861 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-textfile\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.277704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.272867 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.277704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.273076 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e821829-aa45-49a3-a7ef-676c90c57ac2-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.277704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.273307 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.277704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.273671 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-accelerators-collector-config\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.277704 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:42.273918 2574 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 10 18:18:42.277704 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:42.274435 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-state-metrics-tls podName:8e821829-aa45-49a3-a7ef-676c90c57ac2 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:42.774415123 +0000 UTC m=+82.187699902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-lxj76" (UID: "8e821829-aa45-49a3-a7ef-676c90c57ac2") : secret "kube-state-metrics-tls" not found Jul 10 18:18:42.277704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.274886 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-metrics-client-ca\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.277704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.274959 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e821829-aa45-49a3-a7ef-676c90c57ac2-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.277704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.275288 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-tls\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.277704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.275667 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.279919 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.279893 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.284161 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.284130 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnbcg\" (UniqueName: \"kubernetes.io/projected/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-api-access-lnbcg\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.289699 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.289670 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f9r2\" (UniqueName: \"kubernetes.io/projected/d1b45de6-e9ef-409d-82f9-80ec78ffdd27-kube-api-access-5f9r2\") pod \"node-exporter-sh4hv\" (UID: \"d1b45de6-e9ef-409d-82f9-80ec78ffdd27\") " pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.325951 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.325922 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:42.390054 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.389955 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sh4hv" Jul 10 18:18:42.400492 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:18:42.400455 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1b45de6_e9ef_409d_82f9_80ec78ffdd27.slice/crio-d2303ef46156c07161e58ce4bd6db30f0ef47c473258e183f91cfc19a402c56c WatchSource:0}: Error finding container d2303ef46156c07161e58ce4bd6db30f0ef47c473258e183f91cfc19a402c56c: Status 404 returned error can't find the container with id d2303ef46156c07161e58ce4bd6db30f0ef47c473258e183f91cfc19a402c56c Jul 10 18:18:42.427465 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.427429 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:42.527229 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.527198 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:42.778953 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.778921 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.781697 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.781670 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e821829-aa45-49a3-a7ef-676c90c57ac2-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-lxj76\" (UID: \"8e821829-aa45-49a3-a7ef-676c90c57ac2\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:42.782956 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.782923 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sh4hv" event={"ID":"d1b45de6-e9ef-409d-82f9-80ec78ffdd27","Type":"ContainerStarted","Data":"d2303ef46156c07161e58ce4bd6db30f0ef47c473258e183f91cfc19a402c56c"} Jul 10 18:18:42.969996 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:42.969962 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" Jul 10 18:18:43.040154 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.040114 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:43.131265 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.131218 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-lxj76"] Jul 10 18:18:43.137758 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.137734 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:18:43.140540 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.140516 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.141432 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.141389 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:43.142940 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.142923 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 10 18:18:43.143037 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.143013 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 10 18:18:43.143263 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.143248 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 10 18:18:43.143386 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.143266 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 10 18:18:43.143386 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.143322 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 10 18:18:43.143549 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.143511 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 10 18:18:43.143593 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.143552 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 10 18:18:43.143688 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.143610 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 10 18:18:43.143688 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.143656 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5vgt2\"" Jul 10 18:18:43.143846 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.143831 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 10 18:18:43.155878 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.155849 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:18:43.248710 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.248673 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:43.284063 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.283973 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-config-volume\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.284063 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.284026 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.284063 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.284059 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-web-config\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.284330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.284115 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.284330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.284147 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.284330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.284175 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.284330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.284203 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.284330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.284227 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6jdg\" (UniqueName: \"kubernetes.io/projected/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-kube-api-access-p6jdg\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.284330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.284273 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.284330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.284328 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-config-out\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.284648 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.284445 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.284648 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.284493 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.284648 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.284527 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.327208 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:18:43.327163 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e821829_aa45_49a3_a7ef_676c90c57ac2.slice/crio-9a456043c7a80e113d11beda131fcd3118911770932a1e4ad1d4f041512df867 WatchSource:0}: Error finding container 9a456043c7a80e113d11beda131fcd3118911770932a1e4ad1d4f041512df867: Status 404 returned error can't find the container with id 9a456043c7a80e113d11beda131fcd3118911770932a1e4ad1d4f041512df867 Jul 10 18:18:43.385535 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.385501 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.385697 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.385540 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-web-config\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.385697 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.385567 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.385805 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.385731 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.385805 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.385761 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.385805 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.385788 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.385958 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.385932 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.386070 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.386045 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6jdg\" (UniqueName: \"kubernetes.io/projected/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-kube-api-access-p6jdg\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.386178 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.386117 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.386178 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.386146 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-config-out\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.386281 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.386199 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.386281 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.386238 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.386281 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.386273 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.386421 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.386328 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-config-volume\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.388462 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.387236 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.388462 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.388403 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.388840 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.388816 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-web-config\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.389663 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.389476 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-config-volume\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.389837 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.389796 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.390068 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.390042 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.390529 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.390486 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-config-out\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.390646 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.390623 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.391340 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.391313 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.391653 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.391632 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.391730 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.391647 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.394511 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.394487 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6jdg\" (UniqueName: \"kubernetes.io/projected/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-kube-api-access-p6jdg\") pod \"alertmanager-main-0\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.452891 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.452859 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:18:43.459669 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.459636 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:43.517997 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.517962 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:43.600853 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.600818 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:18:43.605248 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:18:43.605221 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e3cb449_f9e5_4ade_8a6f_f40e9d0cfbac.slice/crio-24d7d688a9a681848d6812e38e013aaedde56355aa6ae22a8937081b24692d65 WatchSource:0}: Error finding container 24d7d688a9a681848d6812e38e013aaedde56355aa6ae22a8937081b24692d65: Status 404 returned error can't find the container with id 24d7d688a9a681848d6812e38e013aaedde56355aa6ae22a8937081b24692d65 Jul 10 18:18:43.751746 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.751715 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7nprf" Jul 10 18:18:43.789060 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.788946 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerStarted","Data":"24d7d688a9a681848d6812e38e013aaedde56355aa6ae22a8937081b24692d65"} Jul 10 18:18:43.790496 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.790460 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" event={"ID":"8e821829-aa45-49a3-a7ef-676c90c57ac2","Type":"ContainerStarted","Data":"9a456043c7a80e113d11beda131fcd3118911770932a1e4ad1d4f041512df867"} Jul 10 18:18:43.792348 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.792322 2574 generic.go:356] "Generic (PLEG): container finished" podID="d1b45de6-e9ef-409d-82f9-80ec78ffdd27" containerID="43a54afa898aa6a853938a0eefe820dd621e469868e4dd5dc345b80e265c7912" exitCode=0 Jul 10 18:18:43.792437 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:43.792359 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sh4hv" event={"ID":"d1b45de6-e9ef-409d-82f9-80ec78ffdd27","Type":"ContainerDied","Data":"43a54afa898aa6a853938a0eefe820dd621e469868e4dd5dc345b80e265c7912"} Jul 10 18:18:44.633653 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:44.633550 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:44.733337 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:44.733294 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:44.804960 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:44.804923 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sh4hv" event={"ID":"d1b45de6-e9ef-409d-82f9-80ec78ffdd27","Type":"ContainerStarted","Data":"8ab8e3729e51e176be685535c2db0b476008a2236cadb15d58731eec12c14500"} Jul 10 18:18:44.805136 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:44.805020 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sh4hv" event={"ID":"d1b45de6-e9ef-409d-82f9-80ec78ffdd27","Type":"ContainerStarted","Data":"95fc748544f26a1c967e18c8737127af86279a1781bc6f19f35ed0fb2617fcac"} Jul 10 18:18:44.825079 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:44.825025 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sh4hv" podStartSLOduration=1.795073546 podStartE2EDuration="2.825009703s" totalImagesPullingTime="951.076693ms" totalInitContainerRuntime="78.859464ms" isStatefulPod=true podCreationTimestamp="2026-07-10 18:18:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:44.824367463 +0000 UTC m=+84.237652250" watchObservedRunningTime="2026-07-10 18:18:44.825009703 +0000 UTC m=+84.238294489" Jul 10 18:18:45.013131 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:45.013084 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:45.091323 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:45.091287 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:45.173268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:45.173213 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:45.273836 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:45.273748 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:45.810554 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:45.810519 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerID="6616fb9ca44ea45b1460ce11d28639cad06df9ae66b8ef1ca707af7a08a79cb2" exitCode=0 Jul 10 18:18:45.811041 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:45.810594 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerDied","Data":"6616fb9ca44ea45b1460ce11d28639cad06df9ae66b8ef1ca707af7a08a79cb2"} Jul 10 18:18:45.812837 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:45.812815 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" event={"ID":"8e821829-aa45-49a3-a7ef-676c90c57ac2","Type":"ContainerStarted","Data":"aa1f8b8596e7d742f381281b3d437eb3f75d7018e3fbc5667039c6e28eb2e97a"} Jul 10 18:18:45.812907 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:45.812867 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" event={"ID":"8e821829-aa45-49a3-a7ef-676c90c57ac2","Type":"ContainerStarted","Data":"a546c3328adb2a0fdc495462ff03e8e7ed8834a815b12110556e1ee34c891429"} Jul 10 18:18:45.812907 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:45.812884 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" event={"ID":"8e821829-aa45-49a3-a7ef-676c90c57ac2","Type":"ContainerStarted","Data":"665d6bdf3bc9fb78472240e77f98080998984f52c7e7b1eb392a207610e6ea64"} Jul 10 18:18:45.856661 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:45.856595 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-lxj76" podStartSLOduration=2.17412034 podStartE2EDuration="3.856572371s" totalImagesPullingTime="1.682452031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:18:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:45.856156856 +0000 UTC m=+85.269441643" watchObservedRunningTime="2026-07-10 18:18:45.856572371 +0000 UTC m=+85.269857159" Jul 10 18:18:46.843173 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:46.843142 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz"] Jul 10 18:18:46.843532 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:46.843173 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:46.845450 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:46.845434 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz" Jul 10 18:18:46.848450 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:46.848424 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-9s5px\"" Jul 10 18:18:46.848710 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:46.848691 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 10 18:18:46.854427 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:46.854403 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz"] Jul 10 18:18:46.964298 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:46.964262 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:47.019584 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.019544 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/82fa2512-54af-40f6-9e11-baa115a0bb50-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-6r5zz\" (UID: \"82fa2512-54af-40f6-9e11-baa115a0bb50\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz" Jul 10 18:18:47.019742 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.019622 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxwvq\" (UniqueName: \"kubernetes.io/projected/82fa2512-54af-40f6-9e11-baa115a0bb50-kube-api-access-mxwvq\") pod \"monitoring-plugin-5f68df7f58-6r5zz\" (UID: \"82fa2512-54af-40f6-9e11-baa115a0bb50\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz" Jul 10 18:18:47.078047 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.078002 2574 patch_prober.go:28] interesting pod/image-registry-7cb47549b8-x8zsn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 18:18:47.078255 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.078065 2574 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" podUID="8b5164ee-1735-42c2-8cf1-766f022b5703" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 18:18:47.120559 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.120467 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxwvq\" (UniqueName: \"kubernetes.io/projected/82fa2512-54af-40f6-9e11-baa115a0bb50-kube-api-access-mxwvq\") pod \"monitoring-plugin-5f68df7f58-6r5zz\" (UID: \"82fa2512-54af-40f6-9e11-baa115a0bb50\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz" Jul 10 18:18:47.120559 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.120556 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/82fa2512-54af-40f6-9e11-baa115a0bb50-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-6r5zz\" (UID: \"82fa2512-54af-40f6-9e11-baa115a0bb50\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz" Jul 10 18:18:47.123164 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.123139 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/82fa2512-54af-40f6-9e11-baa115a0bb50-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-6r5zz\" (UID: \"82fa2512-54af-40f6-9e11-baa115a0bb50\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz" Jul 10 18:18:47.133657 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.133622 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxwvq\" (UniqueName: \"kubernetes.io/projected/82fa2512-54af-40f6-9e11-baa115a0bb50-kube-api-access-mxwvq\") pod \"monitoring-plugin-5f68df7f58-6r5zz\" (UID: \"82fa2512-54af-40f6-9e11-baa115a0bb50\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz" Jul 10 18:18:47.154407 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.154373 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz" Jul 10 18:18:47.552380 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.552351 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz"] Jul 10 18:18:47.555241 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:18:47.555212 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82fa2512_54af_40f6_9e11_baa115a0bb50.slice/crio-19953fed14e7db650dbb7b08c6d867bbbf2341da43ea5307aabd2b8213f8a9c5 WatchSource:0}: Error finding container 19953fed14e7db650dbb7b08c6d867bbbf2341da43ea5307aabd2b8213f8a9c5: Status 404 returned error can't find the container with id 19953fed14e7db650dbb7b08c6d867bbbf2341da43ea5307aabd2b8213f8a9c5 Jul 10 18:18:47.823830 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.823793 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz" event={"ID":"82fa2512-54af-40f6-9e11-baa115a0bb50","Type":"ContainerStarted","Data":"19953fed14e7db650dbb7b08c6d867bbbf2341da43ea5307aabd2b8213f8a9c5"} Jul 10 18:18:47.826886 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.826858 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerStarted","Data":"bff5f289453d82edce66485767c875a03e562866b1934215b3d1ea1172a0686e"} Jul 10 18:18:47.826995 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.826918 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerStarted","Data":"1dc92a3fa2d65f277799f1fbfd7315413f1d01396c6cb5af3cc8142fffcef947"} Jul 10 18:18:47.826995 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.826947 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerStarted","Data":"b6086b8dcbd3867677aeee5ae151a255628a44916d7e73632de95e9e970398ff"} Jul 10 18:18:47.826995 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.826973 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerStarted","Data":"28e75e5f59737099e4d272c9e3e6d3369b501ed453fcffe9c77582351ef7a023"} Jul 10 18:18:47.827124 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:47.826995 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerStarted","Data":"a7161fb6a2c6e7d947cb5e0d16206d2629f4a516dbbbe3f5bd31eb7e31b662ee"} Jul 10 18:18:48.310179 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.310147 2574 scope.go:117] "RemoveContainer" containerID="1f74a64d77d94357b354d3622396f0ff40edfb7eebd77aa146453c62ec82ba6b" Jul 10 18:18:48.310632 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:18:48.310343 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-cjjrj_openshift-console-operator(e3770e0a-c127-454b-b0dc-6e4d4a9c1628)\"" pod="openshift-console-operator/console-operator-7866497494-cjjrj" podUID="e3770e0a-c127-454b-b0dc-6e4d4a9c1628" Jul 10 18:18:48.327858 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.327820 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:48.329691 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.329454 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:18:48.333378 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.333356 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.337054 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.336764 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 10 18:18:48.337054 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.337020 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 10 18:18:48.337054 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.337041 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tdbml\"" Jul 10 18:18:48.337310 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.337297 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 10 18:18:48.337368 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.337320 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 10 18:18:48.337938 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.337863 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 10 18:18:48.339151 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.338805 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 10 18:18:48.339151 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.338956 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 10 18:18:48.339529 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.339508 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 10 18:18:48.339884 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.339843 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 10 18:18:48.339884 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.339861 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 10 18:18:48.340427 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.340167 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 10 18:18:48.340521 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.340474 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 10 18:18:48.341278 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.341108 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 10 18:18:48.341278 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.341255 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ufto77ijdvol\"" Jul 10 18:18:48.344022 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.343997 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 10 18:18:48.344444 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.344396 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 10 18:18:48.352725 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.352696 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:18:48.432771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.432722 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f2450c7c-868a-4863-8538-cf6d2228807e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.432942 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.432848 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.432942 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.432887 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.432942 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.432938 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433138 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.432967 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433138 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.432997 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433138 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.433051 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433138 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.433081 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433353 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.433156 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433353 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.433195 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433353 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.433276 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433353 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.433313 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433353 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.433348 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433600 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.433376 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433600 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.433404 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433600 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.433443 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-config\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433600 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.433467 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f2450c7c-868a-4863-8538-cf6d2228807e-config-out\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433600 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.433491 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pfdq\" (UniqueName: \"kubernetes.io/projected/f2450c7c-868a-4863-8538-cf6d2228807e-kube-api-access-8pfdq\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433600 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.433533 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-web-config\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.433600 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.433593 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.534283 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534249 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.534452 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534296 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.534452 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534345 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.534452 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534375 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.534452 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534394 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.534452 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534429 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.534704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534456 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.534704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534483 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.534704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534508 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-config\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.534704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534532 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f2450c7c-868a-4863-8538-cf6d2228807e-config-out\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.534704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534555 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pfdq\" (UniqueName: \"kubernetes.io/projected/f2450c7c-868a-4863-8538-cf6d2228807e-kube-api-access-8pfdq\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.534704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534584 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-web-config\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.534704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534614 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.534704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534663 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f2450c7c-868a-4863-8538-cf6d2228807e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.535086 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534729 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.535086 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534764 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.535086 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534771 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.535086 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534833 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.535086 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534869 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.535086 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534905 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.535086 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534946 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.535086 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.534953 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.535086 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.535032 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.535574 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.535464 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.535638 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.535555 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.537215 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.537035 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.538190 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.537732 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.540130 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.539420 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.540737 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.540664 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-config\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.541060 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.541034 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-web-config\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.543456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.543433 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pfdq\" (UniqueName: \"kubernetes.io/projected/f2450c7c-868a-4863-8538-cf6d2228807e-kube-api-access-8pfdq\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.547963 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.547902 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f2450c7c-868a-4863-8538-cf6d2228807e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.548118 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.548066 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f2450c7c-868a-4863-8538-cf6d2228807e-config-out\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.548177 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.548154 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.548241 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.548185 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.548602 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.548577 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.549507 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.549489 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.550036 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.549984 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.550495 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.550475 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.551655 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.551635 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.648617 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.648516 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:48.708393 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.708365 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:18:48.813826 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.813785 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:18:48.835308 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.835276 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerStarted","Data":"2f8b563cceed9c6e2c1e1253d48060db79012511018b22b54d3def9103f85e2c"} Jul 10 18:18:48.860794 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.860725 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.097881706 podStartE2EDuration="5.860698592s" totalImagesPullingTime="3.743337496s" totalInitContainerRuntime="19.47939ms" isStatefulPod=false podCreationTimestamp="2026-07-10 18:18:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:48.859276541 +0000 UTC m=+88.272561362" watchObservedRunningTime="2026-07-10 18:18:48.860698592 +0000 UTC m=+88.273983382" Jul 10 18:18:48.986518 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:48.986482 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:49.085675 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:49.085636 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:49.132261 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:18:49.132232 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2450c7c_868a_4863_8538_cf6d2228807e.slice/crio-6f3ca5d6f3f836ccdb91f0eb576cb9c236120f504b9aa2034131de7d078e401b WatchSource:0}: Error finding container 6f3ca5d6f3f836ccdb91f0eb576cb9c236120f504b9aa2034131de7d078e401b: Status 404 returned error can't find the container with id 6f3ca5d6f3f836ccdb91f0eb576cb9c236120f504b9aa2034131de7d078e401b Jul 10 18:18:49.841030 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:49.840999 2574 generic.go:356] "Generic (PLEG): container finished" podID="f2450c7c-868a-4863-8538-cf6d2228807e" containerID="a07817e55314c1810ce6223432e28ec60e684f96eebf647cd0c9603f2d5a37b6" exitCode=0 Jul 10 18:18:49.841482 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:49.841091 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerDied","Data":"a07817e55314c1810ce6223432e28ec60e684f96eebf647cd0c9603f2d5a37b6"} Jul 10 18:18:49.841482 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:49.841186 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerStarted","Data":"6f3ca5d6f3f836ccdb91f0eb576cb9c236120f504b9aa2034131de7d078e401b"} Jul 10 18:18:49.842809 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:49.842779 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz" event={"ID":"82fa2512-54af-40f6-9e11-baa115a0bb50","Type":"ContainerStarted","Data":"c7a2079b17a6fd284115941650c207cac3914fdbb1783ec0498f24e40d8a4abf"} Jul 10 18:18:49.843277 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:49.843259 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz" Jul 10 18:18:49.848770 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:49.848748 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz" Jul 10 18:18:49.886238 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:49.886179 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6r5zz" podStartSLOduration=2.285099947 podStartE2EDuration="3.886157168s" totalImagesPullingTime="1.601057221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:49.885932687 +0000 UTC m=+89.299217476" watchObservedRunningTime="2026-07-10 18:18:49.886157168 +0000 UTC m=+89.299441956" Jul 10 18:18:52.620753 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:52.620716 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:52.720998 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:52.720942 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:53.864487 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:53.864396 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerStarted","Data":"9adea74d57e616dc5c2b9146558a9fe199dd3802d500733c7fffe671a21c7666"} Jul 10 18:18:53.864487 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:53.864469 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerStarted","Data":"81ea592fc060a1722181387a9602c1f1f4a1245877504dadd5b79f2c58882f7b"} Jul 10 18:18:54.550031 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:54.549998 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cb47549b8-x8zsn"] Jul 10 18:18:54.618282 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:54.618243 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:54.719676 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:54.719639 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:55.876187 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:55.876084 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerStarted","Data":"9dec7d3804329d672362b9356ca06e2a1b575b5fa5e21753846b6fa0b1476e6e"} Jul 10 18:18:55.876187 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:55.876166 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerStarted","Data":"ffe3dc5cc8732fd641179b4de691d29ec09da93c32cb604cf43ca9c17a7d8223"} Jul 10 18:18:55.876627 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:55.876192 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerStarted","Data":"2dbd489690574fd798ad682f9c76d0c546847828ff5c49a7aed25351c13e7dda"} Jul 10 18:18:55.876627 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:55.876221 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerStarted","Data":"1135a967e214b763595e3bd5d8626c2ea0972247a6b0eefd963d53274c4ce5fb"} Jul 10 18:18:55.906051 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:55.905989 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.547520168 podStartE2EDuration="7.905969696s" totalImagesPullingTime="5.333828451s" totalInitContainerRuntime="24.621077ms" isStatefulPod=false podCreationTimestamp="2026-07-10 18:18:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:55.90442763 +0000 UTC m=+95.317712452" watchObservedRunningTime="2026-07-10 18:18:55.905969696 +0000 UTC m=+95.319254483" Jul 10 18:18:56.376299 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:56.376264 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:56.445193 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:56.445156 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:57.698393 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:57.698362 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-56zsp" Jul 10 18:18:58.649276 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:58.649232 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:18:59.609750 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:59.609720 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:18:59.710755 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:18:59.710723 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:00.160508 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:00.160476 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:00.264145 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:00.264117 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:01.656625 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:01.656591 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:01.755329 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:01.755295 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:02.309833 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:02.309804 2574 scope.go:117] "RemoveContainer" containerID="1f74a64d77d94357b354d3622396f0ff40edfb7eebd77aa146453c62ec82ba6b" Jul 10 18:19:02.907306 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:02.907279 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:19:02.907695 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:02.907357 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-cjjrj" event={"ID":"e3770e0a-c127-454b-b0dc-6e4d4a9c1628","Type":"ContainerStarted","Data":"9e77f643076fcc37fe1bedc067f71306945e6a11d5102145db37ba4c25017d91"} Jul 10 18:19:02.907695 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:02.907642 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:19:02.912323 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:02.912299 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-cjjrj" Jul 10 18:19:02.924527 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:02.924475 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-cjjrj" podStartSLOduration=79.526556659 podStartE2EDuration="1m26.924459307s" totalImagesPullingTime="7.397902648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:19:02.923649147 +0000 UTC m=+102.336933935" watchObservedRunningTime="2026-07-10 18:19:02.924459307 +0000 UTC m=+102.337744095" Jul 10 18:19:03.050081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:03.050041 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:03.146033 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:03.146000 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:03.245280 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:03.245243 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:03.532876 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:03.532793 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:03.719301 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:03.719250 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:03.805367 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:03.805277 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:03.901321 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:03.901288 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:04.133721 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:04.133637 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:04.193052 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:04.193019 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:04.333859 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:04.333822 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:04.435139 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:04.435081 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:04.912421 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:04.912329 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:05.107170 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:05.107088 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:05.258529 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:05.258494 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:05.451702 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:05.451665 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:05.552603 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:05.552518 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:06.725078 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:06.725043 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:06.781614 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:06.781582 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:14.614584 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:14.614544 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:14.715352 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:14.715316 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:15.958727 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:15.958696 2574 generic.go:356] "Generic (PLEG): container finished" podID="d6ade376-624c-4dfc-858b-1940181dfbb3" containerID="241b9656f070a885beb75f0edd0db138e2d17d66bde0e95c0c21042dc5eea589" exitCode=0 Jul 10 18:19:15.959071 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:15.958765 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" event={"ID":"d6ade376-624c-4dfc-858b-1940181dfbb3","Type":"ContainerDied","Data":"241b9656f070a885beb75f0edd0db138e2d17d66bde0e95c0c21042dc5eea589"} Jul 10 18:19:15.959135 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:15.959069 2574 scope.go:117] "RemoveContainer" containerID="241b9656f070a885beb75f0edd0db138e2d17d66bde0e95c0c21042dc5eea589" Jul 10 18:19:16.964568 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:16.964534 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7x8dq" event={"ID":"d6ade376-624c-4dfc-858b-1940181dfbb3","Type":"ContainerStarted","Data":"e4d02147af729586638a73ce479a537b0b8a6f149091fbb646826dd4a56fdd38"} Jul 10 18:19:18.334355 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:18.334320 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:18.407627 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:18.407594 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:19.579888 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.579846 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" podUID="8b5164ee-1735-42c2-8cf1-766f022b5703" containerName="registry" containerID="cri-o://a9addd08ac174f05fca700f60a806457b11701eebb919995f427465b5b9f9079" gracePeriod=30 Jul 10 18:19:19.856119 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.856067 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:19:19.923400 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.923365 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-certificates\") pod \"8b5164ee-1735-42c2-8cf1-766f022b5703\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " Jul 10 18:19:19.923602 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.923421 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b5164ee-1735-42c2-8cf1-766f022b5703-trusted-ca\") pod \"8b5164ee-1735-42c2-8cf1-766f022b5703\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " Jul 10 18:19:19.923602 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.923454 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b5164ee-1735-42c2-8cf1-766f022b5703-ca-trust-extracted\") pod \"8b5164ee-1735-42c2-8cf1-766f022b5703\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " Jul 10 18:19:19.923602 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.923475 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls\") pod \"8b5164ee-1735-42c2-8cf1-766f022b5703\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " Jul 10 18:19:19.923602 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.923497 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-bound-sa-token\") pod \"8b5164ee-1735-42c2-8cf1-766f022b5703\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " Jul 10 18:19:19.923602 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.923529 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6vlb\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-kube-api-access-z6vlb\") pod \"8b5164ee-1735-42c2-8cf1-766f022b5703\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " Jul 10 18:19:19.923602 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.923574 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b5164ee-1735-42c2-8cf1-766f022b5703-image-registry-private-configuration\") pod \"8b5164ee-1735-42c2-8cf1-766f022b5703\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " Jul 10 18:19:19.923903 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.923631 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b5164ee-1735-42c2-8cf1-766f022b5703-installation-pull-secrets\") pod \"8b5164ee-1735-42c2-8cf1-766f022b5703\" (UID: \"8b5164ee-1735-42c2-8cf1-766f022b5703\") " Jul 10 18:19:19.923903 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.923882 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8b5164ee-1735-42c2-8cf1-766f022b5703" (UID: "8b5164ee-1735-42c2-8cf1-766f022b5703"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:19:19.923999 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.923920 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b5164ee-1735-42c2-8cf1-766f022b5703-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8b5164ee-1735-42c2-8cf1-766f022b5703" (UID: "8b5164ee-1735-42c2-8cf1-766f022b5703"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:19:19.926257 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.926198 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b5164ee-1735-42c2-8cf1-766f022b5703-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8b5164ee-1735-42c2-8cf1-766f022b5703" (UID: "8b5164ee-1735-42c2-8cf1-766f022b5703"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:19:19.926599 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.926564 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b5164ee-1735-42c2-8cf1-766f022b5703-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8b5164ee-1735-42c2-8cf1-766f022b5703" (UID: "8b5164ee-1735-42c2-8cf1-766f022b5703"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:19:19.926721 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.926564 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8b5164ee-1735-42c2-8cf1-766f022b5703" (UID: "8b5164ee-1735-42c2-8cf1-766f022b5703"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:19:19.926721 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.926633 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-kube-api-access-z6vlb" (OuterVolumeSpecName: "kube-api-access-z6vlb") pod "8b5164ee-1735-42c2-8cf1-766f022b5703" (UID: "8b5164ee-1735-42c2-8cf1-766f022b5703"). InnerVolumeSpecName "kube-api-access-z6vlb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:19:19.926826 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.926713 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8b5164ee-1735-42c2-8cf1-766f022b5703" (UID: "8b5164ee-1735-42c2-8cf1-766f022b5703"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:19:19.932288 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.932259 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b5164ee-1735-42c2-8cf1-766f022b5703-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8b5164ee-1735-42c2-8cf1-766f022b5703" (UID: "8b5164ee-1735-42c2-8cf1-766f022b5703"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:19:19.977710 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.977677 2574 generic.go:356] "Generic (PLEG): container finished" podID="8b5164ee-1735-42c2-8cf1-766f022b5703" containerID="a9addd08ac174f05fca700f60a806457b11701eebb919995f427465b5b9f9079" exitCode=0 Jul 10 18:19:19.977872 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.977727 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" event={"ID":"8b5164ee-1735-42c2-8cf1-766f022b5703","Type":"ContainerDied","Data":"a9addd08ac174f05fca700f60a806457b11701eebb919995f427465b5b9f9079"} Jul 10 18:19:19.977872 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.977738 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" Jul 10 18:19:19.977872 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.977776 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cb47549b8-x8zsn" event={"ID":"8b5164ee-1735-42c2-8cf1-766f022b5703","Type":"ContainerDied","Data":"5032fbe07ac47d05895229e5b488d796fd2bc3da42a15ac40c3bae22d5aaa19e"} Jul 10 18:19:19.977872 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.977799 2574 scope.go:117] "RemoveContainer" containerID="a9addd08ac174f05fca700f60a806457b11701eebb919995f427465b5b9f9079" Jul 10 18:19:19.987169 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.987075 2574 scope.go:117] "RemoveContainer" containerID="a9addd08ac174f05fca700f60a806457b11701eebb919995f427465b5b9f9079" Jul 10 18:19:19.987431 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:19:19.987413 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9addd08ac174f05fca700f60a806457b11701eebb919995f427465b5b9f9079\": container with ID starting with a9addd08ac174f05fca700f60a806457b11701eebb919995f427465b5b9f9079 not found: ID does not exist" containerID="a9addd08ac174f05fca700f60a806457b11701eebb919995f427465b5b9f9079" Jul 10 18:19:19.987485 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:19.987439 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9addd08ac174f05fca700f60a806457b11701eebb919995f427465b5b9f9079"} err="failed to get container status \"a9addd08ac174f05fca700f60a806457b11701eebb919995f427465b5b9f9079\": rpc error: code = NotFound desc = could not find container \"a9addd08ac174f05fca700f60a806457b11701eebb919995f427465b5b9f9079\": container with ID starting with a9addd08ac174f05fca700f60a806457b11701eebb919995f427465b5b9f9079 not found: ID does not exist" Jul 10 18:19:20.002580 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.002548 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cb47549b8-x8zsn"] Jul 10 18:19:20.008255 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.004906 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7cb47549b8-x8zsn"] Jul 10 18:19:20.025294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.025260 2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b5164ee-1735-42c2-8cf1-766f022b5703-ca-trust-extracted\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:19:20.025294 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.025291 2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-tls\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:19:20.025449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.025302 2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-bound-sa-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:19:20.025449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.025310 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6vlb\" (UniqueName: \"kubernetes.io/projected/8b5164ee-1735-42c2-8cf1-766f022b5703-kube-api-access-z6vlb\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:19:20.025449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.025321 2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b5164ee-1735-42c2-8cf1-766f022b5703-image-registry-private-configuration\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:19:20.025449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.025330 2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b5164ee-1735-42c2-8cf1-766f022b5703-installation-pull-secrets\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:19:20.025449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.025338 2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b5164ee-1735-42c2-8cf1-766f022b5703-registry-certificates\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:19:20.025449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.025346 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b5164ee-1735-42c2-8cf1-766f022b5703-trusted-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:19:20.983014 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.982979 2574 generic.go:356] "Generic (PLEG): container finished" podID="06f82976-1aae-4012-bac5-792e61e2c8af" containerID="47b79530b4d8cbb9eba3c4ef63e29cb0e32db4f654a5068ecee01cdec918bdf9" exitCode=0 Jul 10 18:19:20.983448 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.983053 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" event={"ID":"06f82976-1aae-4012-bac5-792e61e2c8af","Type":"ContainerDied","Data":"47b79530b4d8cbb9eba3c4ef63e29cb0e32db4f654a5068ecee01cdec918bdf9"} Jul 10 18:19:20.983544 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.983525 2574 scope.go:117] "RemoveContainer" containerID="47b79530b4d8cbb9eba3c4ef63e29cb0e32db4f654a5068ecee01cdec918bdf9" Jul 10 18:19:20.984612 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.984592 2574 generic.go:356] "Generic (PLEG): container finished" podID="c81ab235-1f9d-41c8-af87-80ee3aa5c67c" containerID="af81f93869832316e7f54595a49be660770788679af5e71a3d9fc0d6e8474cbf" exitCode=0 Jul 10 18:19:20.984702 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.984659 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" event={"ID":"c81ab235-1f9d-41c8-af87-80ee3aa5c67c","Type":"ContainerDied","Data":"af81f93869832316e7f54595a49be660770788679af5e71a3d9fc0d6e8474cbf"} Jul 10 18:19:20.985020 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:20.985005 2574 scope.go:117] "RemoveContainer" containerID="af81f93869832316e7f54595a49be660770788679af5e71a3d9fc0d6e8474cbf" Jul 10 18:19:21.314837 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:21.314759 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b5164ee-1735-42c2-8cf1-766f022b5703" path="/var/lib/kubelet/pods/8b5164ee-1735-42c2-8cf1-766f022b5703/volumes" Jul 10 18:19:21.991311 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:21.991267 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7dmkh" event={"ID":"c81ab235-1f9d-41c8-af87-80ee3aa5c67c","Type":"ContainerStarted","Data":"0ba3911afae397b5ff21b49362373621b89f3607a06e891dd7f0b3150b4b75ba"} Jul 10 18:19:21.993463 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:21.993436 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-b5h8k" event={"ID":"06f82976-1aae-4012-bac5-792e61e2c8af","Type":"ContainerStarted","Data":"97b0c4279e6c0940b76fa7cf0fd305927fbff4e8d03789789a90547f9e1e8a7f"} Jul 10 18:19:24.933424 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:24.933389 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:24.989357 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:24.989319 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:26.959690 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:26.959657 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:27.026602 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:27.026569 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:31.931646 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:31.931602 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:31.988279 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:31.988247 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:32.501533 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:32.501493 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:48.649785 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:48.649744 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:19:48.666147 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:48.666117 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:19:49.112657 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:49.112629 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:19:59.641554 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:59.641520 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:19:59.735542 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:19:59.735501 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:20:01.623017 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:01.622978 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:20:01.939216 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:01.939181 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:20:02.595625 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:02.595588 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:20:02.595895 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:02.595872 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="alertmanager" containerID="cri-o://a7161fb6a2c6e7d947cb5e0d16206d2629f4a516dbbbe3f5bd31eb7e31b662ee" gracePeriod=120 Jul 10 18:20:02.595964 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:02.595942 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="kube-rbac-proxy-metric" containerID="cri-o://bff5f289453d82edce66485767c875a03e562866b1934215b3d1ea1172a0686e" gracePeriod=120 Jul 10 18:20:02.596013 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:02.595974 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="kube-rbac-proxy" containerID="cri-o://1dc92a3fa2d65f277799f1fbfd7315413f1d01396c6cb5af3cc8142fffcef947" gracePeriod=120 Jul 10 18:20:02.596013 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:02.595965 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="kube-rbac-proxy-web" containerID="cri-o://b6086b8dcbd3867677aeee5ae151a255628a44916d7e73632de95e9e970398ff" gracePeriod=120 Jul 10 18:20:02.596141 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:02.596034 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="prom-label-proxy" containerID="cri-o://2f8b563cceed9c6e2c1e1253d48060db79012511018b22b54d3def9103f85e2c" gracePeriod=120 Jul 10 18:20:02.596141 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:02.596058 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="config-reloader" containerID="cri-o://28e75e5f59737099e4d272c9e3e6d3369b501ed453fcffe9c77582351ef7a023" gracePeriod=120 Jul 10 18:20:03.160254 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.160221 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerID="2f8b563cceed9c6e2c1e1253d48060db79012511018b22b54d3def9103f85e2c" exitCode=0 Jul 10 18:20:03.160254 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.160245 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerID="bff5f289453d82edce66485767c875a03e562866b1934215b3d1ea1172a0686e" exitCode=0 Jul 10 18:20:03.160254 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.160252 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerID="1dc92a3fa2d65f277799f1fbfd7315413f1d01396c6cb5af3cc8142fffcef947" exitCode=0 Jul 10 18:20:03.160254 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.160259 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerID="28e75e5f59737099e4d272c9e3e6d3369b501ed453fcffe9c77582351ef7a023" exitCode=0 Jul 10 18:20:03.160254 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.160266 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerID="a7161fb6a2c6e7d947cb5e0d16206d2629f4a516dbbbe3f5bd31eb7e31b662ee" exitCode=0 Jul 10 18:20:03.160768 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.160294 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerDied","Data":"2f8b563cceed9c6e2c1e1253d48060db79012511018b22b54d3def9103f85e2c"} Jul 10 18:20:03.160768 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.160357 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerDied","Data":"bff5f289453d82edce66485767c875a03e562866b1934215b3d1ea1172a0686e"} Jul 10 18:20:03.160768 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.160383 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerDied","Data":"1dc92a3fa2d65f277799f1fbfd7315413f1d01396c6cb5af3cc8142fffcef947"} Jul 10 18:20:03.160768 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.160407 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerDied","Data":"28e75e5f59737099e4d272c9e3e6d3369b501ed453fcffe9c77582351ef7a023"} Jul 10 18:20:03.160768 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.160430 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerDied","Data":"a7161fb6a2c6e7d947cb5e0d16206d2629f4a516dbbbe3f5bd31eb7e31b662ee"} Jul 10 18:20:03.854128 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.854090 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:03.924777 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.924754 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-main-tls\") pod \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " Jul 10 18:20:03.924955 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.924791 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-alertmanager-trusted-ca-bundle\") pod \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " Jul 10 18:20:03.924955 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.924827 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-metrics-client-ca\") pod \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " Jul 10 18:20:03.924955 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.924850 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-alertmanager-main-db\") pod \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " Jul 10 18:20:03.924955 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.924870 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-web-config\") pod \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " Jul 10 18:20:03.924955 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.924907 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-config-out\") pod \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " Jul 10 18:20:03.924955 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.924953 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6jdg\" (UniqueName: \"kubernetes.io/projected/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-kube-api-access-p6jdg\") pod \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " Jul 10 18:20:03.925287 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.924998 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy-web\") pod \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " Jul 10 18:20:03.925287 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.925038 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-config-volume\") pod \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " Jul 10 18:20:03.925287 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.925067 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-tls-assets\") pod \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " Jul 10 18:20:03.925287 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.925089 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-cluster-tls-config\") pod \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " Jul 10 18:20:03.925287 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.925132 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy-metric\") pod \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " Jul 10 18:20:03.925287 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.925161 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy\") pod \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\" (UID: \"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac\") " Jul 10 18:20:03.925583 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.925315 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" (UID: "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:20:03.925583 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.925363 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" (UID: "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:20:03.925583 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.925520 2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:03.925583 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.925543 2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-alertmanager-main-db\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:03.925849 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.925756 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" (UID: "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:20:03.928359 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.928319 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" (UID: "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:03.929027 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.929001 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" (UID: "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:03.929725 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.929668 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" (UID: "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:03.929725 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.929684 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-config-out" (OuterVolumeSpecName: "config-out") pod "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" (UID: "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:20:03.929725 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.929715 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" (UID: "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:20:03.930172 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.930146 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-kube-api-access-p6jdg" (OuterVolumeSpecName: "kube-api-access-p6jdg") pod "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" (UID: "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac"). InnerVolumeSpecName "kube-api-access-p6jdg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:20:03.930599 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.930576 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" (UID: "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:03.931220 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.931196 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-config-volume" (OuterVolumeSpecName: "config-volume") pod "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" (UID: "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:03.935120 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.935046 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" (UID: "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:03.940899 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:03.940869 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-web-config" (OuterVolumeSpecName: "web-config") pod "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" (UID: "3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:04.030895 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.026597 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-main-tls\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:04.030895 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.026637 2574 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-metrics-client-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:04.030895 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.026665 2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-web-config\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:04.030895 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.026680 2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-config-out\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:04.030895 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.026694 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6jdg\" (UniqueName: \"kubernetes.io/projected/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-kube-api-access-p6jdg\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:04.030895 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.026710 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:04.030895 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.026730 2574 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-config-volume\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:04.030895 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.026743 2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-tls-assets\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:04.030895 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.026755 2574 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-cluster-tls-config\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:04.030895 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.026768 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:04.030895 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.026788 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:04.167011 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.166975 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerID="b6086b8dcbd3867677aeee5ae151a255628a44916d7e73632de95e9e970398ff" exitCode=0 Jul 10 18:20:04.167382 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.167077 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.167382 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.167069 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerDied","Data":"b6086b8dcbd3867677aeee5ae151a255628a44916d7e73632de95e9e970398ff"} Jul 10 18:20:04.167382 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.167222 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac","Type":"ContainerDied","Data":"24d7d688a9a681848d6812e38e013aaedde56355aa6ae22a8937081b24692d65"} Jul 10 18:20:04.167382 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.167274 2574 scope.go:117] "RemoveContainer" containerID="2f8b563cceed9c6e2c1e1253d48060db79012511018b22b54d3def9103f85e2c" Jul 10 18:20:04.177859 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.177831 2574 scope.go:117] "RemoveContainer" containerID="bff5f289453d82edce66485767c875a03e562866b1934215b3d1ea1172a0686e" Jul 10 18:20:04.185228 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.185209 2574 scope.go:117] "RemoveContainer" containerID="1dc92a3fa2d65f277799f1fbfd7315413f1d01396c6cb5af3cc8142fffcef947" Jul 10 18:20:04.192302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.192278 2574 scope.go:117] "RemoveContainer" containerID="b6086b8dcbd3867677aeee5ae151a255628a44916d7e73632de95e9e970398ff" Jul 10 18:20:04.199604 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.199503 2574 scope.go:117] "RemoveContainer" containerID="28e75e5f59737099e4d272c9e3e6d3369b501ed453fcffe9c77582351ef7a023" Jul 10 18:20:04.201598 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.201575 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:20:04.205865 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.205844 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:20:04.208485 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.208462 2574 scope.go:117] "RemoveContainer" containerID="a7161fb6a2c6e7d947cb5e0d16206d2629f4a516dbbbe3f5bd31eb7e31b662ee" Jul 10 18:20:04.215745 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.215726 2574 scope.go:117] "RemoveContainer" containerID="6616fb9ca44ea45b1460ce11d28639cad06df9ae66b8ef1ca707af7a08a79cb2" Jul 10 18:20:04.222933 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.222912 2574 scope.go:117] "RemoveContainer" containerID="2f8b563cceed9c6e2c1e1253d48060db79012511018b22b54d3def9103f85e2c" Jul 10 18:20:04.223206 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:20:04.223188 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f8b563cceed9c6e2c1e1253d48060db79012511018b22b54d3def9103f85e2c\": container with ID starting with 2f8b563cceed9c6e2c1e1253d48060db79012511018b22b54d3def9103f85e2c not found: ID does not exist" containerID="2f8b563cceed9c6e2c1e1253d48060db79012511018b22b54d3def9103f85e2c" Jul 10 18:20:04.223254 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.223217 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f8b563cceed9c6e2c1e1253d48060db79012511018b22b54d3def9103f85e2c"} err="failed to get container status \"2f8b563cceed9c6e2c1e1253d48060db79012511018b22b54d3def9103f85e2c\": rpc error: code = NotFound desc = could not find container \"2f8b563cceed9c6e2c1e1253d48060db79012511018b22b54d3def9103f85e2c\": container with ID starting with 2f8b563cceed9c6e2c1e1253d48060db79012511018b22b54d3def9103f85e2c not found: ID does not exist" Jul 10 18:20:04.223254 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.223238 2574 scope.go:117] "RemoveContainer" containerID="bff5f289453d82edce66485767c875a03e562866b1934215b3d1ea1172a0686e" Jul 10 18:20:04.223476 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:20:04.223456 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bff5f289453d82edce66485767c875a03e562866b1934215b3d1ea1172a0686e\": container with ID starting with bff5f289453d82edce66485767c875a03e562866b1934215b3d1ea1172a0686e not found: ID does not exist" containerID="bff5f289453d82edce66485767c875a03e562866b1934215b3d1ea1172a0686e" Jul 10 18:20:04.223515 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.223484 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bff5f289453d82edce66485767c875a03e562866b1934215b3d1ea1172a0686e"} err="failed to get container status \"bff5f289453d82edce66485767c875a03e562866b1934215b3d1ea1172a0686e\": rpc error: code = NotFound desc = could not find container \"bff5f289453d82edce66485767c875a03e562866b1934215b3d1ea1172a0686e\": container with ID starting with bff5f289453d82edce66485767c875a03e562866b1934215b3d1ea1172a0686e not found: ID does not exist" Jul 10 18:20:04.223515 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.223501 2574 scope.go:117] "RemoveContainer" containerID="1dc92a3fa2d65f277799f1fbfd7315413f1d01396c6cb5af3cc8142fffcef947" Jul 10 18:20:04.223744 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:20:04.223727 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dc92a3fa2d65f277799f1fbfd7315413f1d01396c6cb5af3cc8142fffcef947\": container with ID starting with 1dc92a3fa2d65f277799f1fbfd7315413f1d01396c6cb5af3cc8142fffcef947 not found: ID does not exist" containerID="1dc92a3fa2d65f277799f1fbfd7315413f1d01396c6cb5af3cc8142fffcef947" Jul 10 18:20:04.223822 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.223749 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dc92a3fa2d65f277799f1fbfd7315413f1d01396c6cb5af3cc8142fffcef947"} err="failed to get container status \"1dc92a3fa2d65f277799f1fbfd7315413f1d01396c6cb5af3cc8142fffcef947\": rpc error: code = NotFound desc = could not find container \"1dc92a3fa2d65f277799f1fbfd7315413f1d01396c6cb5af3cc8142fffcef947\": container with ID starting with 1dc92a3fa2d65f277799f1fbfd7315413f1d01396c6cb5af3cc8142fffcef947 not found: ID does not exist" Jul 10 18:20:04.223822 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.223765 2574 scope.go:117] "RemoveContainer" containerID="b6086b8dcbd3867677aeee5ae151a255628a44916d7e73632de95e9e970398ff" Jul 10 18:20:04.223990 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:20:04.223971 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6086b8dcbd3867677aeee5ae151a255628a44916d7e73632de95e9e970398ff\": container with ID starting with b6086b8dcbd3867677aeee5ae151a255628a44916d7e73632de95e9e970398ff not found: ID does not exist" containerID="b6086b8dcbd3867677aeee5ae151a255628a44916d7e73632de95e9e970398ff" Jul 10 18:20:04.224030 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.223993 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6086b8dcbd3867677aeee5ae151a255628a44916d7e73632de95e9e970398ff"} err="failed to get container status \"b6086b8dcbd3867677aeee5ae151a255628a44916d7e73632de95e9e970398ff\": rpc error: code = NotFound desc = could not find container \"b6086b8dcbd3867677aeee5ae151a255628a44916d7e73632de95e9e970398ff\": container with ID starting with b6086b8dcbd3867677aeee5ae151a255628a44916d7e73632de95e9e970398ff not found: ID does not exist" Jul 10 18:20:04.224030 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.224007 2574 scope.go:117] "RemoveContainer" containerID="28e75e5f59737099e4d272c9e3e6d3369b501ed453fcffe9c77582351ef7a023" Jul 10 18:20:04.224260 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:20:04.224241 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28e75e5f59737099e4d272c9e3e6d3369b501ed453fcffe9c77582351ef7a023\": container with ID starting with 28e75e5f59737099e4d272c9e3e6d3369b501ed453fcffe9c77582351ef7a023 not found: ID does not exist" containerID="28e75e5f59737099e4d272c9e3e6d3369b501ed453fcffe9c77582351ef7a023" Jul 10 18:20:04.224327 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.224264 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28e75e5f59737099e4d272c9e3e6d3369b501ed453fcffe9c77582351ef7a023"} err="failed to get container status \"28e75e5f59737099e4d272c9e3e6d3369b501ed453fcffe9c77582351ef7a023\": rpc error: code = NotFound desc = could not find container \"28e75e5f59737099e4d272c9e3e6d3369b501ed453fcffe9c77582351ef7a023\": container with ID starting with 28e75e5f59737099e4d272c9e3e6d3369b501ed453fcffe9c77582351ef7a023 not found: ID does not exist" Jul 10 18:20:04.224327 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.224279 2574 scope.go:117] "RemoveContainer" containerID="a7161fb6a2c6e7d947cb5e0d16206d2629f4a516dbbbe3f5bd31eb7e31b662ee" Jul 10 18:20:04.224513 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:20:04.224493 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7161fb6a2c6e7d947cb5e0d16206d2629f4a516dbbbe3f5bd31eb7e31b662ee\": container with ID starting with a7161fb6a2c6e7d947cb5e0d16206d2629f4a516dbbbe3f5bd31eb7e31b662ee not found: ID does not exist" containerID="a7161fb6a2c6e7d947cb5e0d16206d2629f4a516dbbbe3f5bd31eb7e31b662ee" Jul 10 18:20:04.224555 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.224514 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7161fb6a2c6e7d947cb5e0d16206d2629f4a516dbbbe3f5bd31eb7e31b662ee"} err="failed to get container status \"a7161fb6a2c6e7d947cb5e0d16206d2629f4a516dbbbe3f5bd31eb7e31b662ee\": rpc error: code = NotFound desc = could not find container \"a7161fb6a2c6e7d947cb5e0d16206d2629f4a516dbbbe3f5bd31eb7e31b662ee\": container with ID starting with a7161fb6a2c6e7d947cb5e0d16206d2629f4a516dbbbe3f5bd31eb7e31b662ee not found: ID does not exist" Jul 10 18:20:04.224555 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.224529 2574 scope.go:117] "RemoveContainer" containerID="6616fb9ca44ea45b1460ce11d28639cad06df9ae66b8ef1ca707af7a08a79cb2" Jul 10 18:20:04.224754 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:20:04.224736 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6616fb9ca44ea45b1460ce11d28639cad06df9ae66b8ef1ca707af7a08a79cb2\": container with ID starting with 6616fb9ca44ea45b1460ce11d28639cad06df9ae66b8ef1ca707af7a08a79cb2 not found: ID does not exist" containerID="6616fb9ca44ea45b1460ce11d28639cad06df9ae66b8ef1ca707af7a08a79cb2" Jul 10 18:20:04.224820 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.224761 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6616fb9ca44ea45b1460ce11d28639cad06df9ae66b8ef1ca707af7a08a79cb2"} err="failed to get container status \"6616fb9ca44ea45b1460ce11d28639cad06df9ae66b8ef1ca707af7a08a79cb2\": rpc error: code = NotFound desc = could not find container \"6616fb9ca44ea45b1460ce11d28639cad06df9ae66b8ef1ca707af7a08a79cb2\": container with ID starting with 6616fb9ca44ea45b1460ce11d28639cad06df9ae66b8ef1ca707af7a08a79cb2 not found: ID does not exist" Jul 10 18:20:04.256452 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256416 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:20:04.256737 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256724 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="config-reloader" Jul 10 18:20:04.256786 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256738 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="config-reloader" Jul 10 18:20:04.256786 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256750 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="kube-rbac-proxy" Jul 10 18:20:04.256786 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256756 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="kube-rbac-proxy" Jul 10 18:20:04.256786 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256763 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="prom-label-proxy" Jul 10 18:20:04.256786 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256768 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="prom-label-proxy" Jul 10 18:20:04.256786 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256778 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="kube-rbac-proxy-metric" Jul 10 18:20:04.256786 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256785 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="kube-rbac-proxy-metric" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256794 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="kube-rbac-proxy-web" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256800 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="kube-rbac-proxy-web" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256807 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="init-config-reloader" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256814 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="init-config-reloader" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256822 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b5164ee-1735-42c2-8cf1-766f022b5703" containerName="registry" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256828 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b5164ee-1735-42c2-8cf1-766f022b5703" containerName="registry" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256834 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="alertmanager" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256839 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="alertmanager" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256891 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="kube-rbac-proxy-metric" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256899 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="kube-rbac-proxy-web" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256905 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="prom-label-proxy" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256910 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b5164ee-1735-42c2-8cf1-766f022b5703" containerName="registry" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256915 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="kube-rbac-proxy" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256920 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="alertmanager" Jul 10 18:20:04.256989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.256927 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" containerName="config-reloader" Jul 10 18:20:04.260599 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.260583 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.274638 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.274615 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 10 18:20:04.278884 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.278869 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 10 18:20:04.282496 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.282482 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 10 18:20:04.282543 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.282504 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 10 18:20:04.287404 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.287383 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 10 18:20:04.294946 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.294928 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5vgt2\"" Jul 10 18:20:04.295018 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.294937 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 10 18:20:04.295170 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.295156 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 10 18:20:04.311389 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.311367 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 10 18:20:04.328775 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.328749 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/229c3dcc-b6f5-4c24-8230-cd427d420cbe-config-out\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.328872 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.328782 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cxh7\" (UniqueName: \"kubernetes.io/projected/229c3dcc-b6f5-4c24-8230-cd427d420cbe-kube-api-access-7cxh7\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.328872 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.328802 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/229c3dcc-b6f5-4c24-8230-cd427d420cbe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.328872 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.328853 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/229c3dcc-b6f5-4c24-8230-cd427d420cbe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.328982 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.328894 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-config-volume\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.328982 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.328916 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.328982 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.328940 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.329118 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.328984 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.329118 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.329016 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/229c3dcc-b6f5-4c24-8230-cd427d420cbe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.329118 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.329044 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-web-config\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.329118 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.329067 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.329244 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.329175 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/229c3dcc-b6f5-4c24-8230-cd427d420cbe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.329244 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.329199 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.335214 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.335188 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:20:04.339908 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.339885 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 10 18:20:04.430389 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.430302 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/229c3dcc-b6f5-4c24-8230-cd427d420cbe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.430389 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.430343 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-config-volume\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.430389 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.430362 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.430389 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.430382 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.430707 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.430415 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.430707 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.430436 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/229c3dcc-b6f5-4c24-8230-cd427d420cbe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.430707 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.430474 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-web-config\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.430864 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.430708 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.430864 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.430764 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/229c3dcc-b6f5-4c24-8230-cd427d420cbe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.430864 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.430810 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.431012 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.430887 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/229c3dcc-b6f5-4c24-8230-cd427d420cbe-config-out\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.431012 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.430916 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cxh7\" (UniqueName: \"kubernetes.io/projected/229c3dcc-b6f5-4c24-8230-cd427d420cbe-kube-api-access-7cxh7\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.431012 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.430948 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/229c3dcc-b6f5-4c24-8230-cd427d420cbe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.431197 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.431178 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/229c3dcc-b6f5-4c24-8230-cd427d420cbe-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.431680 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.431358 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/229c3dcc-b6f5-4c24-8230-cd427d420cbe-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.432237 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.432174 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/229c3dcc-b6f5-4c24-8230-cd427d420cbe-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.433652 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.433626 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-config-volume\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.433758 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.433633 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-web-config\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.433912 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.433869 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.434020 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.434000 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/229c3dcc-b6f5-4c24-8230-cd427d420cbe-tls-assets\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.434084 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.434063 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.434322 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.434304 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.434536 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.434518 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.434609 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.434552 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/229c3dcc-b6f5-4c24-8230-cd427d420cbe-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.435089 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.435071 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/229c3dcc-b6f5-4c24-8230-cd427d420cbe-config-out\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.440675 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.440658 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cxh7\" (UniqueName: \"kubernetes.io/projected/229c3dcc-b6f5-4c24-8230-cd427d420cbe-kube-api-access-7cxh7\") pod \"alertmanager-main-0\" (UID: \"229c3dcc-b6f5-4c24-8230-cd427d420cbe\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.569752 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.569715 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 18:20:04.716175 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:04.716082 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 18:20:04.719752 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:20:04.719725 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod229c3dcc_b6f5_4c24_8230_cd427d420cbe.slice/crio-d10d621c4e7c37da09f9b851643732d82cf469546bfb9b66ef3d77bf269f8ec1 WatchSource:0}: Error finding container d10d621c4e7c37da09f9b851643732d82cf469546bfb9b66ef3d77bf269f8ec1: Status 404 returned error can't find the container with id d10d621c4e7c37da09f9b851643732d82cf469546bfb9b66ef3d77bf269f8ec1 Jul 10 18:20:05.173133 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:05.173033 2574 generic.go:356] "Generic (PLEG): container finished" podID="229c3dcc-b6f5-4c24-8230-cd427d420cbe" containerID="ee324176d72897f8b5f053ba67eae858b26c3cd9b46e61d3f5128057ec7f3495" exitCode=0 Jul 10 18:20:05.173586 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:05.173128 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"229c3dcc-b6f5-4c24-8230-cd427d420cbe","Type":"ContainerDied","Data":"ee324176d72897f8b5f053ba67eae858b26c3cd9b46e61d3f5128057ec7f3495"} Jul 10 18:20:05.173586 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:05.173209 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"229c3dcc-b6f5-4c24-8230-cd427d420cbe","Type":"ContainerStarted","Data":"d10d621c4e7c37da09f9b851643732d82cf469546bfb9b66ef3d77bf269f8ec1"} Jul 10 18:20:05.313797 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:05.313767 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac" path="/var/lib/kubelet/pods/3e3cb449-f9e5-4ade-8a6f-f40e9d0cfbac/volumes" Jul 10 18:20:06.187402 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.187360 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"229c3dcc-b6f5-4c24-8230-cd427d420cbe","Type":"ContainerStarted","Data":"e8debe86de59e11d414922bdb370d57e9697a970dfe2e8d1e492fff0287e20bb"} Jul 10 18:20:06.187768 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.187448 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"229c3dcc-b6f5-4c24-8230-cd427d420cbe","Type":"ContainerStarted","Data":"b52f8cdb2232c1e81e5de33d1d7d7f804daa1f97044c87c5c3f60445703bf1f5"} Jul 10 18:20:06.187768 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.187475 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"229c3dcc-b6f5-4c24-8230-cd427d420cbe","Type":"ContainerStarted","Data":"677cdcab81de2f8f6ea2c5342f2ce09eb18a058023d2408fcfee7a2865718788"} Jul 10 18:20:06.187768 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.187501 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"229c3dcc-b6f5-4c24-8230-cd427d420cbe","Type":"ContainerStarted","Data":"3f15859a1185135a9076fc69cfa578518b5d958ec13208488edfcdeb69b07a48"} Jul 10 18:20:06.187768 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.187523 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"229c3dcc-b6f5-4c24-8230-cd427d420cbe","Type":"ContainerStarted","Data":"f16534e73030fa2f68fc19c35c4392879ba4635640a04be98db5121b6287bb97"} Jul 10 18:20:06.187768 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.187544 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"229c3dcc-b6f5-4c24-8230-cd427d420cbe","Type":"ContainerStarted","Data":"db11ca5621175cb8dae770bfa205a1cd12f0f4ea39594b3ebd1529539a555e83"} Jul 10 18:20:06.216992 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.216938 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.200108237 podStartE2EDuration="2.216921602s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.813365ms" isStatefulPod=false podCreationTimestamp="2026-07-10 18:20:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:20:06.213960243 +0000 UTC m=+165.627245052" watchObservedRunningTime="2026-07-10 18:20:06.216921602 +0000 UTC m=+165.630206388" Jul 10 18:20:06.726388 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.726351 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb"] Jul 10 18:20:06.730327 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.730298 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.732768 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.732740 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 10 18:20:06.732923 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.732808 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 10 18:20:06.732923 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.732842 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-mkgjp\"" Jul 10 18:20:06.732923 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.732808 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 10 18:20:06.733073 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.732995 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 10 18:20:06.733131 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.733069 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 10 18:20:06.738076 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.738051 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 10 18:20:06.743139 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.743113 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb"] Jul 10 18:20:06.855029 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.854998 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-serving-certs-ca-bundle\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.855243 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.855055 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-telemeter-client-tls\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.855243 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.855134 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.855243 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.855156 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.855243 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.855206 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blgqv\" (UniqueName: \"kubernetes.io/projected/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-kube-api-access-blgqv\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.855402 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.855247 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-federate-client-tls\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.855402 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.855276 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-secret-telemeter-client\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.855402 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.855302 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-metrics-client-ca\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.956257 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.956216 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-serving-certs-ca-bundle\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.956440 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.956285 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-telemeter-client-tls\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.956440 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.956309 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.956440 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.956325 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.956615 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.956435 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blgqv\" (UniqueName: \"kubernetes.io/projected/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-kube-api-access-blgqv\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.956615 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.956480 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-federate-client-tls\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.956615 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.956522 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-secret-telemeter-client\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.956615 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.956551 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-metrics-client-ca\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.957172 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.957143 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-serving-certs-ca-bundle\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.957301 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.957266 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.957301 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.957276 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-metrics-client-ca\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.958942 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.958913 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.959059 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.959037 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-secret-telemeter-client\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.959264 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.959249 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-telemeter-client-tls\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.959300 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.959278 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-federate-client-tls\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:06.965131 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:06.965070 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blgqv\" (UniqueName: \"kubernetes.io/projected/56a7f6c5-3b0f-417e-8acf-cc12d9871a6f-kube-api-access-blgqv\") pod \"telemeter-client-7d56fbf68d-9qzzb\" (UID: \"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f\") " pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:07.037393 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.037309 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:20:07.037650 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.037592 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="prometheus" containerID="cri-o://81ea592fc060a1722181387a9602c1f1f4a1245877504dadd5b79f2c58882f7b" gracePeriod=600 Jul 10 18:20:07.037793 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.037639 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="kube-rbac-proxy-web" containerID="cri-o://2dbd489690574fd798ad682f9c76d0c546847828ff5c49a7aed25351c13e7dda" gracePeriod=600 Jul 10 18:20:07.037793 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.037669 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="kube-rbac-proxy-thanos" containerID="cri-o://9dec7d3804329d672362b9356ca06e2a1b575b5fa5e21753846b6fa0b1476e6e" gracePeriod=600 Jul 10 18:20:07.037793 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.037634 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="thanos-sidecar" containerID="cri-o://1135a967e214b763595e3bd5d8626c2ea0972247a6b0eefd963d53274c4ce5fb" gracePeriod=600 Jul 10 18:20:07.037793 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.037618 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="kube-rbac-proxy" containerID="cri-o://ffe3dc5cc8732fd641179b4de691d29ec09da93c32cb604cf43ca9c17a7d8223" gracePeriod=600 Jul 10 18:20:07.037793 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.037652 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="config-reloader" containerID="cri-o://9adea74d57e616dc5c2b9146558a9fe199dd3802d500733c7fffe671a21c7666" gracePeriod=600 Jul 10 18:20:07.040359 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.040333 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" Jul 10 18:20:07.188901 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.188869 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb"] Jul 10 18:20:07.191373 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:20:07.191340 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a7f6c5_3b0f_417e_8acf_cc12d9871a6f.slice/crio-2e54f18c4469b4ce9a0873749158ac822bae9a0aae287ed9caf329aa88219b97 WatchSource:0}: Error finding container 2e54f18c4469b4ce9a0873749158ac822bae9a0aae287ed9caf329aa88219b97: Status 404 returned error can't find the container with id 2e54f18c4469b4ce9a0873749158ac822bae9a0aae287ed9caf329aa88219b97 Jul 10 18:20:07.200746 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.200719 2574 generic.go:356] "Generic (PLEG): container finished" podID="f2450c7c-868a-4863-8538-cf6d2228807e" containerID="9dec7d3804329d672362b9356ca06e2a1b575b5fa5e21753846b6fa0b1476e6e" exitCode=0 Jul 10 18:20:07.200746 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.200743 2574 generic.go:356] "Generic (PLEG): container finished" podID="f2450c7c-868a-4863-8538-cf6d2228807e" containerID="ffe3dc5cc8732fd641179b4de691d29ec09da93c32cb604cf43ca9c17a7d8223" exitCode=0 Jul 10 18:20:07.200746 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.200750 2574 generic.go:356] "Generic (PLEG): container finished" podID="f2450c7c-868a-4863-8538-cf6d2228807e" containerID="2dbd489690574fd798ad682f9c76d0c546847828ff5c49a7aed25351c13e7dda" exitCode=0 Jul 10 18:20:07.200935 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.200756 2574 generic.go:356] "Generic (PLEG): container finished" podID="f2450c7c-868a-4863-8538-cf6d2228807e" containerID="1135a967e214b763595e3bd5d8626c2ea0972247a6b0eefd963d53274c4ce5fb" exitCode=0 Jul 10 18:20:07.200935 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.200761 2574 generic.go:356] "Generic (PLEG): container finished" podID="f2450c7c-868a-4863-8538-cf6d2228807e" containerID="9adea74d57e616dc5c2b9146558a9fe199dd3802d500733c7fffe671a21c7666" exitCode=0 Jul 10 18:20:07.200935 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.200766 2574 generic.go:356] "Generic (PLEG): container finished" podID="f2450c7c-868a-4863-8538-cf6d2228807e" containerID="81ea592fc060a1722181387a9602c1f1f4a1245877504dadd5b79f2c58882f7b" exitCode=0 Jul 10 18:20:07.200935 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.200753 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerDied","Data":"9dec7d3804329d672362b9356ca06e2a1b575b5fa5e21753846b6fa0b1476e6e"} Jul 10 18:20:07.200935 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.200917 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerDied","Data":"ffe3dc5cc8732fd641179b4de691d29ec09da93c32cb604cf43ca9c17a7d8223"} Jul 10 18:20:07.201204 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.200963 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerDied","Data":"2dbd489690574fd798ad682f9c76d0c546847828ff5c49a7aed25351c13e7dda"} Jul 10 18:20:07.201204 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.201005 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerDied","Data":"1135a967e214b763595e3bd5d8626c2ea0972247a6b0eefd963d53274c4ce5fb"} Jul 10 18:20:07.201204 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.201048 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerDied","Data":"9adea74d57e616dc5c2b9146558a9fe199dd3802d500733c7fffe671a21c7666"} Jul 10 18:20:07.201204 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.201089 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerDied","Data":"81ea592fc060a1722181387a9602c1f1f4a1245877504dadd5b79f2c58882f7b"} Jul 10 18:20:07.284305 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.284280 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:07.359357 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.359284 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-metrics-client-certs\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.359357 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.359316 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-db\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.359357 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.359337 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-metrics-client-ca\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.359676 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.359378 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-kube-rbac-proxy\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.359676 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.359406 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-trusted-ca-bundle\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.359676 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.359428 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-2\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360070 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.359871 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:20:07.360070 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.359875 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:20:07.360070 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.359462 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-config\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360070 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.359980 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360070 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360000 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f2450c7c-868a-4863-8538-cf6d2228807e-config-out\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360355 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360180 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360355 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360223 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-grpc-tls\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360355 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360253 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-thanos-prometheus-http-client-file\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360355 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360290 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f2450c7c-868a-4863-8538-cf6d2228807e-tls-assets\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360355 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360311 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:20:07.360355 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360330 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-kubelet-serving-ca-bundle\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360661 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360356 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-0\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360661 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360391 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-1\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360661 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360417 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-serving-certs-ca-bundle\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360661 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360443 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-tls\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360661 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360474 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-web-config\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360661 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360521 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pfdq\" (UniqueName: \"kubernetes.io/projected/f2450c7c-868a-4863-8538-cf6d2228807e-kube-api-access-8pfdq\") pod \"f2450c7c-868a-4863-8538-cf6d2228807e\" (UID: \"f2450c7c-868a-4863-8538-cf6d2228807e\") " Jul 10 18:20:07.360951 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360779 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:20:07.361002 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.360964 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:20:07.361599 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.361334 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:20:07.361755 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.361727 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:20:07.362143 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.362088 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:07.362836 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.362705 2574 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-metrics-client-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.362836 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.362732 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-db\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.362836 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.362749 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-metrics-client-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.363680 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.363148 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:20:07.363680 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.363542 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.363680 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.363568 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.363680 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.363583 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.363680 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.363603 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.363680 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.363618 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.364527 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.364492 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:07.364660 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.364634 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:07.364880 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.364832 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:07.364880 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.364832 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2450c7c-868a-4863-8538-cf6d2228807e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:20:07.365033 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.365004 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:07.365258 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.365240 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-config" (OuterVolumeSpecName: "config") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:07.365356 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.365333 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:07.365356 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.365344 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2450c7c-868a-4863-8538-cf6d2228807e-config-out" (OuterVolumeSpecName: "config-out") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:20:07.365583 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.365567 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:07.366686 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.366650 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2450c7c-868a-4863-8538-cf6d2228807e-kube-api-access-8pfdq" (OuterVolumeSpecName: "kube-api-access-8pfdq") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "kube-api-access-8pfdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:20:07.376509 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.376486 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-web-config" (OuterVolumeSpecName: "web-config") pod "f2450c7c-868a-4863-8538-cf6d2228807e" (UID: "f2450c7c-868a-4863-8538-cf6d2228807e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:20:07.464785 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.464742 2574 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-config\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.464785 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.464779 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.464785 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.464789 2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f2450c7c-868a-4863-8538-cf6d2228807e-config-out\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.465015 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.464799 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.465015 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.464808 2574 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-grpc-tls\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.465015 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.464817 2574 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.465015 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.464827 2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f2450c7c-868a-4863-8538-cf6d2228807e-tls-assets\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.465015 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.464836 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f2450c7c-868a-4863-8538-cf6d2228807e-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.465015 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.464845 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.465015 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.464853 2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-web-config\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.465015 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.464861 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pfdq\" (UniqueName: \"kubernetes.io/projected/f2450c7c-868a-4863-8538-cf6d2228807e-kube-api-access-8pfdq\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:07.465015 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:07.464869 2574 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f2450c7c-868a-4863-8538-cf6d2228807e-secret-kube-rbac-proxy\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:20:08.207046 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.207011 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" event={"ID":"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f","Type":"ContainerStarted","Data":"2e54f18c4469b4ce9a0873749158ac822bae9a0aae287ed9caf329aa88219b97"} Jul 10 18:20:08.210120 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.210080 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f2450c7c-868a-4863-8538-cf6d2228807e","Type":"ContainerDied","Data":"6f3ca5d6f3f836ccdb91f0eb576cb9c236120f504b9aa2034131de7d078e401b"} Jul 10 18:20:08.210243 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.210141 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.210243 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.210162 2574 scope.go:117] "RemoveContainer" containerID="9dec7d3804329d672362b9356ca06e2a1b575b5fa5e21753846b6fa0b1476e6e" Jul 10 18:20:08.218329 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.218306 2574 scope.go:117] "RemoveContainer" containerID="ffe3dc5cc8732fd641179b4de691d29ec09da93c32cb604cf43ca9c17a7d8223" Jul 10 18:20:08.228620 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.228595 2574 scope.go:117] "RemoveContainer" containerID="2dbd489690574fd798ad682f9c76d0c546847828ff5c49a7aed25351c13e7dda" Jul 10 18:20:08.235883 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.235856 2574 scope.go:117] "RemoveContainer" containerID="1135a967e214b763595e3bd5d8626c2ea0972247a6b0eefd963d53274c4ce5fb" Jul 10 18:20:08.237590 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.237566 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:20:08.243302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.243283 2574 scope.go:117] "RemoveContainer" containerID="9adea74d57e616dc5c2b9146558a9fe199dd3802d500733c7fffe671a21c7666" Jul 10 18:20:08.246922 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.246870 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:20:08.251474 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.251451 2574 scope.go:117] "RemoveContainer" containerID="81ea592fc060a1722181387a9602c1f1f4a1245877504dadd5b79f2c58882f7b" Jul 10 18:20:08.258944 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.258917 2574 scope.go:117] "RemoveContainer" containerID="a07817e55314c1810ce6223432e28ec60e684f96eebf647cd0c9603f2d5a37b6" Jul 10 18:20:08.276832 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.276809 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:20:08.277160 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277148 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="kube-rbac-proxy" Jul 10 18:20:08.277202 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277162 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="kube-rbac-proxy" Jul 10 18:20:08.277202 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277172 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="config-reloader" Jul 10 18:20:08.277202 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277178 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="config-reloader" Jul 10 18:20:08.277202 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277189 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="kube-rbac-proxy-thanos" Jul 10 18:20:08.277202 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277194 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="kube-rbac-proxy-thanos" Jul 10 18:20:08.277202 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277201 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="thanos-sidecar" Jul 10 18:20:08.277373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277207 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="thanos-sidecar" Jul 10 18:20:08.277373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277216 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="prometheus" Jul 10 18:20:08.277373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277222 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="prometheus" Jul 10 18:20:08.277373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277230 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="kube-rbac-proxy-web" Jul 10 18:20:08.277373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277235 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="kube-rbac-proxy-web" Jul 10 18:20:08.277373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277243 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="init-config-reloader" Jul 10 18:20:08.277373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277248 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="init-config-reloader" Jul 10 18:20:08.277373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277304 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="thanos-sidecar" Jul 10 18:20:08.277373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277311 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="kube-rbac-proxy-thanos" Jul 10 18:20:08.277373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277317 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="prometheus" Jul 10 18:20:08.277373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277323 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="config-reloader" Jul 10 18:20:08.277373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277328 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="kube-rbac-proxy-web" Jul 10 18:20:08.277373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.277333 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" containerName="kube-rbac-proxy" Jul 10 18:20:08.282470 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.282453 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.285981 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.285955 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 10 18:20:08.286148 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.285955 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 10 18:20:08.286331 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.286311 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 10 18:20:08.286782 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.286575 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 10 18:20:08.286782 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.286594 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 10 18:20:08.286782 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.286612 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 10 18:20:08.286782 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.286628 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ufto77ijdvol\"" Jul 10 18:20:08.286782 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.286648 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tdbml\"" Jul 10 18:20:08.286782 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.286628 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 10 18:20:08.286782 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.286677 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 10 18:20:08.286782 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.286615 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 10 18:20:08.287158 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.287032 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 10 18:20:08.287158 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.287133 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 10 18:20:08.287311 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.287256 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 10 18:20:08.287311 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.287296 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 10 18:20:08.288572 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.288554 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 10 18:20:08.292451 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.292429 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 10 18:20:08.306372 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.306344 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:20:08.375616 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.375581 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.375781 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.375629 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.375781 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.375695 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-config\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.375781 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.375725 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64vdp\" (UniqueName: \"kubernetes.io/projected/531c296a-3a99-47dd-91dc-fa754f1523f4-kube-api-access-64vdp\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.375781 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.375743 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.375781 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.375769 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.375997 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.375791 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.375997 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.375809 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.375997 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.375827 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.375997 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.375891 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.375997 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.375913 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.376255 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.376022 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/531c296a-3a99-47dd-91dc-fa754f1523f4-config-out\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.376255 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.376063 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.376255 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.376109 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/531c296a-3a99-47dd-91dc-fa754f1523f4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.376255 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.376145 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-web-config\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.376255 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.376178 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.376255 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.376218 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.376255 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.376246 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.376492 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.376268 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.376492 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.376339 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477131 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477006 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477131 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477050 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/531c296a-3a99-47dd-91dc-fa754f1523f4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477131 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477069 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-web-config\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477406 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477155 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477406 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477197 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477406 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477225 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477406 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477250 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477406 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477295 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477406 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477335 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477406 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477368 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477406 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477403 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-config\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477443 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64vdp\" (UniqueName: \"kubernetes.io/projected/531c296a-3a99-47dd-91dc-fa754f1523f4-kube-api-access-64vdp\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477472 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477508 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477536 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477565 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477592 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477635 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477662 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.477771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477695 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/531c296a-3a99-47dd-91dc-fa754f1523f4-config-out\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.478234 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.477911 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.478698 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.478392 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.479176 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.478859 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.480576 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.480244 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-web-config\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.480576 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.480271 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/531c296a-3a99-47dd-91dc-fa754f1523f4-config-out\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.480576 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.480372 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.480576 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.480452 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.480863 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.480747 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.480863 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.480813 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.480965 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.480906 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.481566 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.481259 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/531c296a-3a99-47dd-91dc-fa754f1523f4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.481659 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.481579 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.481659 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.481599 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-config\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.481764 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.481654 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.481764 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.481684 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.483218 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.483196 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.483496 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.483471 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.483573 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.483548 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/531c296a-3a99-47dd-91dc-fa754f1523f4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.484258 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.484238 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/531c296a-3a99-47dd-91dc-fa754f1523f4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.489524 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.489503 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64vdp\" (UniqueName: \"kubernetes.io/projected/531c296a-3a99-47dd-91dc-fa754f1523f4-kube-api-access-64vdp\") pod \"prometheus-k8s-0\" (UID: \"531c296a-3a99-47dd-91dc-fa754f1523f4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.593152 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.593113 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:08.745645 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:08.745576 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 18:20:08.748706 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:20:08.748679 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod531c296a_3a99_47dd_91dc_fa754f1523f4.slice/crio-50a4f2997d57e89ac284f7c02caa9e3e88ea547d74db69586f14ce94ae3f7988 WatchSource:0}: Error finding container 50a4f2997d57e89ac284f7c02caa9e3e88ea547d74db69586f14ce94ae3f7988: Status 404 returned error can't find the container with id 50a4f2997d57e89ac284f7c02caa9e3e88ea547d74db69586f14ce94ae3f7988 Jul 10 18:20:09.216882 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:09.216849 2574 generic.go:356] "Generic (PLEG): container finished" podID="531c296a-3a99-47dd-91dc-fa754f1523f4" containerID="09bdabaf5d2e5bcd9e0c21b2b3086a669d8c41025b042acf8d0ede6d3c4116b6" exitCode=0 Jul 10 18:20:09.217325 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:09.216940 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"531c296a-3a99-47dd-91dc-fa754f1523f4","Type":"ContainerDied","Data":"09bdabaf5d2e5bcd9e0c21b2b3086a669d8c41025b042acf8d0ede6d3c4116b6"} Jul 10 18:20:09.217325 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:09.217021 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"531c296a-3a99-47dd-91dc-fa754f1523f4","Type":"ContainerStarted","Data":"50a4f2997d57e89ac284f7c02caa9e3e88ea547d74db69586f14ce94ae3f7988"} Jul 10 18:20:09.316822 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:09.316783 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2450c7c-868a-4863-8538-cf6d2228807e" path="/var/lib/kubelet/pods/f2450c7c-868a-4863-8538-cf6d2228807e/volumes" Jul 10 18:20:10.224181 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:10.224139 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" event={"ID":"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f","Type":"ContainerStarted","Data":"d96780f71ea9805fbb2c65f54972ed4a7d1ca194d9f596cf0a2988e6cd9dc68c"} Jul 10 18:20:10.224628 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:10.224240 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" event={"ID":"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f","Type":"ContainerStarted","Data":"4ea5bb2f04f7e307525b0f3720d7d6cea0fbd213c9041654caa28c0f31af45f0"} Jul 10 18:20:10.224628 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:10.224270 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" event={"ID":"56a7f6c5-3b0f-417e-8acf-cc12d9871a6f","Type":"ContainerStarted","Data":"4fb960bbcbbc652df0117ddab3c8ea0d1a10ed5ba84c6263eb983d65688a6bd7"} Jul 10 18:20:10.227290 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:10.227266 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"531c296a-3a99-47dd-91dc-fa754f1523f4","Type":"ContainerStarted","Data":"f375a9f944fca5b069d03d2a296cf893cf706d10454b85c6df44dfba7da0efe9"} Jul 10 18:20:10.227412 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:10.227331 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"531c296a-3a99-47dd-91dc-fa754f1523f4","Type":"ContainerStarted","Data":"bd3fe0ed654bfbf22840e307a988694d6a8b06fb239b1d2dfe8150655cdffef8"} Jul 10 18:20:10.227412 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:10.227373 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"531c296a-3a99-47dd-91dc-fa754f1523f4","Type":"ContainerStarted","Data":"1ce969b67ffd2b1c6519288565ab5311c9be884d7f2b955e362d4306b3270a2c"} Jul 10 18:20:10.227537 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:10.227415 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"531c296a-3a99-47dd-91dc-fa754f1523f4","Type":"ContainerStarted","Data":"6dc6bcc3ae1bbb0c10caa04fead0d9c10c584fc6f30a38f142a5a3d019c65a80"} Jul 10 18:20:10.227537 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:10.227454 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"531c296a-3a99-47dd-91dc-fa754f1523f4","Type":"ContainerStarted","Data":"209c182aa890940260b2f0b6f8ce6b8fba66bd4d406c0c9809aec687ee7c91ab"} Jul 10 18:20:10.227537 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:10.227503 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"531c296a-3a99-47dd-91dc-fa754f1523f4","Type":"ContainerStarted","Data":"cdce001b6534f4d98ec6666fe28c9c9e52277d82f9e9cfc911fb0ec1421a5879"} Jul 10 18:20:10.246953 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:10.246895 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7d56fbf68d-9qzzb" podStartSLOduration=1.814536494 podStartE2EDuration="4.246878545s" totalImagesPullingTime="2.432342051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:20:10.245126586 +0000 UTC m=+169.658411367" watchObservedRunningTime="2026-07-10 18:20:10.246878545 +0000 UTC m=+169.660163331" Jul 10 18:20:10.273536 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:10.273483 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.252543033 podStartE2EDuration="2.273463889s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.920856ms" isStatefulPod=false podCreationTimestamp="2026-07-10 18:20:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:20:10.272293888 +0000 UTC m=+169.685578678" watchObservedRunningTime="2026-07-10 18:20:10.273463889 +0000 UTC m=+169.686748676" Jul 10 18:20:11.219622 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:11.219587 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:20:11.320234 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:11.320203 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:20:13.593698 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:13.593599 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:20:22.136218 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:22.136186 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:20:22.238604 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:22.238567 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:20:22.339069 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:20:22.339032 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:08.593397 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:08.593360 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:21:08.609466 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:08.609440 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:21:09.481638 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:09.481612 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 18:21:16.826389 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:16.826351 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:16.882328 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:16.882296 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:26.595532 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:26.595498 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:26.695140 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:26.695089 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:26.796039 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:26.796008 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:27.302656 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:27.302602 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:27.383697 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:27.383666 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:28.827177 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:28.827142 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:28.930576 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:28.930546 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:37.417409 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:37.417374 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:37.519209 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:37.519174 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:37.573074 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:37.573039 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:40.533211 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:40.533166 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:44.665917 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:44.665810 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:21:44.768230 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:21:44.768198 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:19.080481 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:19.080442 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:20.418511 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:20.418451 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:21.064917 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:21.064890 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:22:21.068990 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:21.068959 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:22:21.071716 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:21.071691 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:22:21.074922 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:21.074899 2574 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 18:22:21.076872 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:21.076849 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:22:40.731528 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:40.731499 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:41.793001 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:41.792964 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:44.536636 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:44.536597 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:44.638580 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:44.638532 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:46.771078 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:46.771030 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:46.931063 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:46.931020 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:47.490262 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:47.490214 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:47.613976 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:47.613941 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:49.151856 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:49.151818 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:49.387531 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:49.387496 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:51.330970 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:51.330938 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:51.390542 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:51.390505 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:56.364828 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:56.364798 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:22:57.750702 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:22:57.750663 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:03.503918 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:03.503882 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:03.602003 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:03.601964 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:09.651302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:09.651235 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:09.752139 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:09.752072 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:11.139653 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:11.139620 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:11.202648 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:11.202614 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:11.680841 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:11.680806 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcm79"] Jul 10 18:23:11.684337 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:11.684322 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcm79" Jul 10 18:23:11.686941 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:11.686922 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:23:11.687508 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:11.687489 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 18:23:11.688018 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:11.688002 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-mnzx7\"" Jul 10 18:23:11.695373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:11.695342 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcm79"] Jul 10 18:23:11.782109 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:11.782073 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24035f6b-02a5-4d95-afbb-a8e1bf528334-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kcm79\" (UID: \"24035f6b-02a5-4d95-afbb-a8e1bf528334\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcm79" Jul 10 18:23:11.782291 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:11.782174 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gn67\" (UniqueName: \"kubernetes.io/projected/24035f6b-02a5-4d95-afbb-a8e1bf528334-kube-api-access-6gn67\") pod \"openshift-lws-operator-fd8ccff4c-kcm79\" (UID: \"24035f6b-02a5-4d95-afbb-a8e1bf528334\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcm79" Jul 10 18:23:11.883416 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:11.883374 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24035f6b-02a5-4d95-afbb-a8e1bf528334-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kcm79\" (UID: \"24035f6b-02a5-4d95-afbb-a8e1bf528334\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcm79" Jul 10 18:23:11.883416 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:11.883418 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gn67\" (UniqueName: \"kubernetes.io/projected/24035f6b-02a5-4d95-afbb-a8e1bf528334-kube-api-access-6gn67\") pod \"openshift-lws-operator-fd8ccff4c-kcm79\" (UID: \"24035f6b-02a5-4d95-afbb-a8e1bf528334\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcm79" Jul 10 18:23:11.883818 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:11.883792 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24035f6b-02a5-4d95-afbb-a8e1bf528334-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kcm79\" (UID: \"24035f6b-02a5-4d95-afbb-a8e1bf528334\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcm79" Jul 10 18:23:11.897053 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:11.897021 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gn67\" (UniqueName: \"kubernetes.io/projected/24035f6b-02a5-4d95-afbb-a8e1bf528334-kube-api-access-6gn67\") pod \"openshift-lws-operator-fd8ccff4c-kcm79\" (UID: \"24035f6b-02a5-4d95-afbb-a8e1bf528334\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcm79" Jul 10 18:23:12.008780 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:12.008683 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcm79" Jul 10 18:23:12.141073 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:12.141011 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcm79"] Jul 10 18:23:12.148385 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:12.148361 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:23:12.985753 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:12.985719 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcm79" event={"ID":"24035f6b-02a5-4d95-afbb-a8e1bf528334","Type":"ContainerStarted","Data":"0a343965878e602bc77c6e665e9a221820cc47cb92aced2daaf99517cc6423d0"} Jul 10 18:23:14.996841 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:14.996804 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcm79" event={"ID":"24035f6b-02a5-4d95-afbb-a8e1bf528334","Type":"ContainerStarted","Data":"b64387c85af155a8a55b79d85c266403fd478c33d639e0142918ba1d35f6d194"} Jul 10 18:23:15.019707 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:15.019643 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcm79" podStartSLOduration=1.6369142490000002 podStartE2EDuration="4.019623159s" totalImagesPullingTime="2.38270891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:23:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:23:15.016823384 +0000 UTC m=+354.430108166" watchObservedRunningTime="2026-07-10 18:23:15.019623159 +0000 UTC m=+354.432907950" Jul 10 18:23:15.628355 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:15.628320 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:15.728493 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:15.728444 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:17.742803 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:17.742769 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:27.980704 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:27.980666 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:28.043041 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.043006 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n"] Jul 10 18:23:28.046358 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.046342 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:28.051209 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.051186 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-95bfc\"" Jul 10 18:23:28.053200 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.053055 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 10 18:23:28.054648 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.054629 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 10 18:23:28.056947 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.056928 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 10 18:23:28.076372 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.076342 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n"] Jul 10 18:23:28.112301 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.112267 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:28.130445 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.130415 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53b8d0b4-91d8-4e6d-a74c-7342d69edd26-cert\") pod \"lws-controller-manager-6cfc4d774-stv4n\" (UID: \"53b8d0b4-91d8-4e6d-a74c-7342d69edd26\") " pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:28.130575 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.130465 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/53b8d0b4-91d8-4e6d-a74c-7342d69edd26-metrics-cert\") pod \"lws-controller-manager-6cfc4d774-stv4n\" (UID: \"53b8d0b4-91d8-4e6d-a74c-7342d69edd26\") " pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:28.130575 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.130517 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/53b8d0b4-91d8-4e6d-a74c-7342d69edd26-manager-config\") pod \"lws-controller-manager-6cfc4d774-stv4n\" (UID: \"53b8d0b4-91d8-4e6d-a74c-7342d69edd26\") " pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:28.130575 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.130549 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-694gq\" (UniqueName: \"kubernetes.io/projected/53b8d0b4-91d8-4e6d-a74c-7342d69edd26-kube-api-access-694gq\") pod \"lws-controller-manager-6cfc4d774-stv4n\" (UID: \"53b8d0b4-91d8-4e6d-a74c-7342d69edd26\") " pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:28.231220 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.231082 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53b8d0b4-91d8-4e6d-a74c-7342d69edd26-cert\") pod \"lws-controller-manager-6cfc4d774-stv4n\" (UID: \"53b8d0b4-91d8-4e6d-a74c-7342d69edd26\") " pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:28.231220 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.231183 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/53b8d0b4-91d8-4e6d-a74c-7342d69edd26-metrics-cert\") pod \"lws-controller-manager-6cfc4d774-stv4n\" (UID: \"53b8d0b4-91d8-4e6d-a74c-7342d69edd26\") " pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:28.231220 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.231222 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/53b8d0b4-91d8-4e6d-a74c-7342d69edd26-manager-config\") pod \"lws-controller-manager-6cfc4d774-stv4n\" (UID: \"53b8d0b4-91d8-4e6d-a74c-7342d69edd26\") " pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:28.231512 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.231265 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-694gq\" (UniqueName: \"kubernetes.io/projected/53b8d0b4-91d8-4e6d-a74c-7342d69edd26-kube-api-access-694gq\") pod \"lws-controller-manager-6cfc4d774-stv4n\" (UID: \"53b8d0b4-91d8-4e6d-a74c-7342d69edd26\") " pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:28.231907 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.231884 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/53b8d0b4-91d8-4e6d-a74c-7342d69edd26-manager-config\") pod \"lws-controller-manager-6cfc4d774-stv4n\" (UID: \"53b8d0b4-91d8-4e6d-a74c-7342d69edd26\") " pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:28.233544 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.233515 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/53b8d0b4-91d8-4e6d-a74c-7342d69edd26-metrics-cert\") pod \"lws-controller-manager-6cfc4d774-stv4n\" (UID: \"53b8d0b4-91d8-4e6d-a74c-7342d69edd26\") " pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:28.233663 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.233545 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53b8d0b4-91d8-4e6d-a74c-7342d69edd26-cert\") pod \"lws-controller-manager-6cfc4d774-stv4n\" (UID: \"53b8d0b4-91d8-4e6d-a74c-7342d69edd26\") " pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:28.241551 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.241533 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-694gq\" (UniqueName: \"kubernetes.io/projected/53b8d0b4-91d8-4e6d-a74c-7342d69edd26-kube-api-access-694gq\") pod \"lws-controller-manager-6cfc4d774-stv4n\" (UID: \"53b8d0b4-91d8-4e6d-a74c-7342d69edd26\") " pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:28.356228 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.356195 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:28.491748 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:28.491721 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n"] Jul 10 18:23:28.493992 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:23:28.493964 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53b8d0b4_91d8_4e6d_a74c_7342d69edd26.slice/crio-15cbbe4e67e9990758ad95471791d8bee697159dd4be8e7874b405077d02d20b WatchSource:0}: Error finding container 15cbbe4e67e9990758ad95471791d8bee697159dd4be8e7874b405077d02d20b: Status 404 returned error can't find the container with id 15cbbe4e67e9990758ad95471791d8bee697159dd4be8e7874b405077d02d20b Jul 10 18:23:29.067690 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:29.067656 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" event={"ID":"53b8d0b4-91d8-4e6d-a74c-7342d69edd26","Type":"ContainerStarted","Data":"15cbbe4e67e9990758ad95471791d8bee697159dd4be8e7874b405077d02d20b"} Jul 10 18:23:31.077595 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:31.077558 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" event={"ID":"53b8d0b4-91d8-4e6d-a74c-7342d69edd26","Type":"ContainerStarted","Data":"3afa3ef7ed5950357bef158b05445389d929824e575888abecd6ec2d18b16bae"} Jul 10 18:23:31.078036 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:31.077688 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:33.596362 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:33.596327 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:33.692021 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:33.691982 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:34.260372 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.260339 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:34.508747 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.508709 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:34.613723 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.613634 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:34.663124 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.659089 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" podStartSLOduration=5.003849847 podStartE2EDuration="6.659065314s" totalImagesPullingTime="1.655215467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:23:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:23:31.098785565 +0000 UTC m=+370.512070353" watchObservedRunningTime="2026-07-10 18:23:34.659065314 +0000 UTC m=+374.072350091" Jul 10 18:23:34.663124 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.661329 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4"] Jul 10 18:23:34.666260 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.666228 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.668748 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.668719 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 18:23:34.668891 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.668720 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 18:23:34.668891 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.668800 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 18:23:34.669207 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.669183 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5dt62\"" Jul 10 18:23:34.669207 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.669198 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 18:23:34.691492 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.691455 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/003942ce-485d-48ee-9261-5f72541c940f-tmp\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.691492 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.691494 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/003942ce-485d-48ee-9261-5f72541c940f-manifests\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.691697 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.691549 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/003942ce-485d-48ee-9261-5f72541c940f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.691697 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.691582 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/003942ce-485d-48ee-9261-5f72541c940f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.691697 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.691601 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trqrf\" (UniqueName: \"kubernetes.io/projected/003942ce-485d-48ee-9261-5f72541c940f-kube-api-access-trqrf\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.694622 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.694593 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4"] Jul 10 18:23:34.779620 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.779588 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:34.792114 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.792070 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/003942ce-485d-48ee-9261-5f72541c940f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.792289 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.792130 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trqrf\" (UniqueName: \"kubernetes.io/projected/003942ce-485d-48ee-9261-5f72541c940f-kube-api-access-trqrf\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.792289 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.792186 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/003942ce-485d-48ee-9261-5f72541c940f-tmp\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.792289 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.792214 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/003942ce-485d-48ee-9261-5f72541c940f-manifests\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.792289 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.792272 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/003942ce-485d-48ee-9261-5f72541c940f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.792682 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.792658 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/003942ce-485d-48ee-9261-5f72541c940f-manifests\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.794605 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.794579 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/003942ce-485d-48ee-9261-5f72541c940f-tmp\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.794765 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.794750 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/003942ce-485d-48ee-9261-5f72541c940f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.794807 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.794770 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/003942ce-485d-48ee-9261-5f72541c940f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.807840 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.807808 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trqrf\" (UniqueName: \"kubernetes.io/projected/003942ce-485d-48ee-9261-5f72541c940f-kube-api-access-trqrf\") pod \"opendatahub-operator-controller-manager-6d75465dc8-9xgl4\" (UID: \"003942ce-485d-48ee-9261-5f72541c940f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:34.914649 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.914556 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:34.977741 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:34.977708 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:35.007154 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:35.007124 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:35.071546 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:35.071512 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:35.118609 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:35.118577 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4"] Jul 10 18:23:35.186008 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:35.185978 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:35.273791 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:35.273757 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:35.371449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:35.371413 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:36.101450 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:36.101411 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" event={"ID":"003942ce-485d-48ee-9261-5f72541c940f","Type":"ContainerStarted","Data":"3b1e5003a78d91ed8f3a39d78d8b09d3c4af012de3bdc9c7c4fa0f40df1d1e84"} Jul 10 18:23:39.120635 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:39.120602 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" event={"ID":"003942ce-485d-48ee-9261-5f72541c940f","Type":"ContainerStarted","Data":"fd85a51ba241f7edd283ec8de89585de570805bc65116e7c3131e5cde2124fbe"} Jul 10 18:23:40.126041 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:40.126000 2574 generic.go:356] "Generic (PLEG): container finished" podID="003942ce-485d-48ee-9261-5f72541c940f" containerID="fd85a51ba241f7edd283ec8de89585de570805bc65116e7c3131e5cde2124fbe" exitCode=0 Jul 10 18:23:40.126511 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:40.126117 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" event={"ID":"003942ce-485d-48ee-9261-5f72541c940f","Type":"ContainerDied","Data":"fd85a51ba241f7edd283ec8de89585de570805bc65116e7c3131e5cde2124fbe"} Jul 10 18:23:41.133506 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:41.133476 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" event={"ID":"003942ce-485d-48ee-9261-5f72541c940f","Type":"ContainerStarted","Data":"2aabadd09588672a5f226716d582ae9d36c78442d50851ec41ecf5baf85a6ede"} Jul 10 18:23:41.133900 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:41.133607 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:41.159191 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:41.159144 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" podStartSLOduration=2.502407512 podStartE2EDuration="7.15912812s" totalImagesPullingTime="3.512399225s" totalInitContainerRuntime="1.144321383s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:23:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:23:41.156701165 +0000 UTC m=+380.569985953" watchObservedRunningTime="2026-07-10 18:23:41.15912812 +0000 UTC m=+380.572412906" Jul 10 18:23:42.083867 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:42.083836 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6cfc4d774-stv4n" Jul 10 18:23:46.207507 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:46.207465 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:46.643246 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:46.643159 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:46.704078 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:46.704046 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:52.140526 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:52.140493 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-9xgl4" Jul 10 18:23:54.068883 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:54.068849 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:54.204760 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:54.204726 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:54.815434 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:54.815398 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:54.960138 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:54.960080 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99"] Jul 10 18:23:54.966893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:54.966873 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" Jul 10 18:23:54.969444 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:54.969415 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 18:23:54.969845 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:54.969828 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mgsjt\"" Jul 10 18:23:54.970792 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:54.970774 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 18:23:54.981000 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:54.980961 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99"] Jul 10 18:23:55.062080 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.062023 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:55.076381 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.076287 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/445ffa19-7b40-4f32-8e19-fb30f8f045bd-tls-certs\") pod \"kube-auth-proxy-79bb7c45b8-qtp99\" (UID: \"445ffa19-7b40-4f32-8e19-fb30f8f045bd\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" Jul 10 18:23:55.076381 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.076345 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/445ffa19-7b40-4f32-8e19-fb30f8f045bd-tmp\") pod \"kube-auth-proxy-79bb7c45b8-qtp99\" (UID: \"445ffa19-7b40-4f32-8e19-fb30f8f045bd\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" Jul 10 18:23:55.076381 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.076369 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qstdl\" (UniqueName: \"kubernetes.io/projected/445ffa19-7b40-4f32-8e19-fb30f8f045bd-kube-api-access-qstdl\") pod \"kube-auth-proxy-79bb7c45b8-qtp99\" (UID: \"445ffa19-7b40-4f32-8e19-fb30f8f045bd\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" Jul 10 18:23:55.164682 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.164648 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:55.177860 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.177825 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/445ffa19-7b40-4f32-8e19-fb30f8f045bd-tls-certs\") pod \"kube-auth-proxy-79bb7c45b8-qtp99\" (UID: \"445ffa19-7b40-4f32-8e19-fb30f8f045bd\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" Jul 10 18:23:55.177987 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.177884 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/445ffa19-7b40-4f32-8e19-fb30f8f045bd-tmp\") pod \"kube-auth-proxy-79bb7c45b8-qtp99\" (UID: \"445ffa19-7b40-4f32-8e19-fb30f8f045bd\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" Jul 10 18:23:55.177987 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.177908 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qstdl\" (UniqueName: \"kubernetes.io/projected/445ffa19-7b40-4f32-8e19-fb30f8f045bd-kube-api-access-qstdl\") pod \"kube-auth-proxy-79bb7c45b8-qtp99\" (UID: \"445ffa19-7b40-4f32-8e19-fb30f8f045bd\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" Jul 10 18:23:55.178129 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:23:55.177990 2574 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 10 18:23:55.178129 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:23:55.178083 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/445ffa19-7b40-4f32-8e19-fb30f8f045bd-tls-certs podName:445ffa19-7b40-4f32-8e19-fb30f8f045bd nodeName:}" failed. No retries permitted until 2026-07-10 18:23:55.678059845 +0000 UTC m=+395.091344627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/445ffa19-7b40-4f32-8e19-fb30f8f045bd-tls-certs") pod "kube-auth-proxy-79bb7c45b8-qtp99" (UID: "445ffa19-7b40-4f32-8e19-fb30f8f045bd") : secret "kube-auth-proxy-tls" not found Jul 10 18:23:55.180226 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.180203 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/445ffa19-7b40-4f32-8e19-fb30f8f045bd-tmp\") pod \"kube-auth-proxy-79bb7c45b8-qtp99\" (UID: \"445ffa19-7b40-4f32-8e19-fb30f8f045bd\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" Jul 10 18:23:55.190817 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.190789 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qstdl\" (UniqueName: \"kubernetes.io/projected/445ffa19-7b40-4f32-8e19-fb30f8f045bd-kube-api-access-qstdl\") pod \"kube-auth-proxy-79bb7c45b8-qtp99\" (UID: \"445ffa19-7b40-4f32-8e19-fb30f8f045bd\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" Jul 10 18:23:55.306008 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.305971 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:55.410132 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.410008 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:55.518199 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.518156 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:55.682750 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.682714 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/445ffa19-7b40-4f32-8e19-fb30f8f045bd-tls-certs\") pod \"kube-auth-proxy-79bb7c45b8-qtp99\" (UID: \"445ffa19-7b40-4f32-8e19-fb30f8f045bd\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" Jul 10 18:23:55.685109 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.685065 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/445ffa19-7b40-4f32-8e19-fb30f8f045bd-tls-certs\") pod \"kube-auth-proxy-79bb7c45b8-qtp99\" (UID: \"445ffa19-7b40-4f32-8e19-fb30f8f045bd\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" Jul 10 18:23:55.832400 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.832369 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:55.878230 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:55.878198 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" Jul 10 18:23:56.036811 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:56.036774 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99"] Jul 10 18:23:56.041508 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:23:56.041473 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod445ffa19_7b40_4f32_8e19_fb30f8f045bd.slice/crio-f2f909c50e6c013de74d9179f7f88ab2558e1860150606379da159e436be5e3b WatchSource:0}: Error finding container f2f909c50e6c013de74d9179f7f88ab2558e1860150606379da159e436be5e3b: Status 404 returned error can't find the container with id f2f909c50e6c013de74d9179f7f88ab2558e1860150606379da159e436be5e3b Jul 10 18:23:56.127510 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:56.127477 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:56.212645 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:56.212614 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" event={"ID":"445ffa19-7b40-4f32-8e19-fb30f8f045bd","Type":"ContainerStarted","Data":"f2f909c50e6c013de74d9179f7f88ab2558e1860150606379da159e436be5e3b"} Jul 10 18:23:56.326425 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:56.326336 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:56.405937 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:56.405907 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:23:57.355247 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:23:57.355210 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:00.235891 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:00.235853 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" event={"ID":"445ffa19-7b40-4f32-8e19-fb30f8f045bd","Type":"ContainerStarted","Data":"5000c1d0812d485ae57ab886d32c0fa80dbbc0d805f16eb0a1a491e478d9a463"} Jul 10 18:24:00.252731 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:00.252679 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-qtp99" podStartSLOduration=2.996164335 podStartE2EDuration="6.252662394s" totalImagesPullingTime="3.256498059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:24:00.250371777 +0000 UTC m=+399.663656563" watchObservedRunningTime="2026-07-10 18:24:00.252662394 +0000 UTC m=+399.665947181" Jul 10 18:24:02.339245 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:02.339213 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:02.413810 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:02.413773 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:02.630731 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:02.630653 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:02.835206 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:02.835170 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:02.926216 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:02.926179 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:04.561066 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:04.561032 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:04.667983 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:04.667947 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:05.378057 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:05.378024 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:05.598044 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:05.598008 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:05.733534 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:05.733491 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:05.841456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:05.841410 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:06.755484 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:06.755452 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:06.856274 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:06.856236 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:06.954250 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:06.954216 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:07.016978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:07.016893 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:09.369241 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:09.369204 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:09.448987 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:09.448945 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:29.632056 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:29.632021 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:29.682382 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:29.682351 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:38.682910 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:38.682875 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:38.739246 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:38.739215 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:55.411776 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:55.411692 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:55.509234 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:55.509197 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:24:55.593303 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:24:55.593273 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:12.068418 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:12.068378 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:12.167727 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:12.167693 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:12.269279 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:12.269246 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:13.011693 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:13.011658 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:13.132149 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:13.132120 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:23.442471 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:23.442431 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:23.539885 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:23.539848 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:23.601428 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:23.601397 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:27.084042 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:27.084007 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:30.472637 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:30.472599 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:31.282282 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:31.282243 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:34.975013 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:34.974975 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:35.122050 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:35.122017 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:37.402323 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:37.402290 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:37.744167 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:37.744085 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:38.154067 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:38.153980 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:40.126864 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:40.126835 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:43.125337 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:43.125299 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:43.401023 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:43.400948 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:48.561529 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:48.561492 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:48.868654 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:48.868572 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:48.969495 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:48.969463 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:52.053811 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:52.053779 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:52.278737 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:52.278708 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:52.382218 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:52.382091 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:52.709900 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:52.709862 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:56.079538 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:56.079504 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:25:56.180273 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:25:56.180241 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:03.046045 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:03.046012 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:03.149239 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:03.149198 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:05.742161 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:05.742130 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:05.841157 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:05.841120 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:10.878601 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:10.878560 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:10.938567 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:10.938536 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:11.523488 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:11.523452 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:11.643398 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:11.643368 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:14.905718 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:14.905634 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:15.059389 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:15.059359 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:15.151615 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:15.151581 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:18.101874 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:18.101819 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:18.203679 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:18.203645 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:35.046349 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:35.046320 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:35.470377 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:35.470342 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:52.989483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:52.989436 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:53.087768 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.087732 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:53.187182 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.187153 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:53.357601 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.357516 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:26:53.359976 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.359955 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-tgmjb" Jul 10 18:26:53.363364 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.363345 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 18:26:53.363909 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.363888 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 18:26:53.364036 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.363939 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 18:26:53.364036 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.363954 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-c6frz\"" Jul 10 18:26:53.370351 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.370330 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:26:53.404564 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.404527 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:26:53.461111 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.461069 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74r99\" (UniqueName: \"kubernetes.io/projected/ffb6f688-9ecd-4c59-bdcc-707407d60eb2-kube-api-access-74r99\") pod \"limitador-limitador-54d6d988d5-tgmjb\" (UID: \"ffb6f688-9ecd-4c59-bdcc-707407d60eb2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tgmjb" Jul 10 18:26:53.461290 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.461180 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ffb6f688-9ecd-4c59-bdcc-707407d60eb2-config-file\") pod \"limitador-limitador-54d6d988d5-tgmjb\" (UID: \"ffb6f688-9ecd-4c59-bdcc-707407d60eb2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tgmjb" Jul 10 18:26:53.562304 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.562265 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74r99\" (UniqueName: \"kubernetes.io/projected/ffb6f688-9ecd-4c59-bdcc-707407d60eb2-kube-api-access-74r99\") pod \"limitador-limitador-54d6d988d5-tgmjb\" (UID: \"ffb6f688-9ecd-4c59-bdcc-707407d60eb2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tgmjb" Jul 10 18:26:53.562474 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.562318 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ffb6f688-9ecd-4c59-bdcc-707407d60eb2-config-file\") pod \"limitador-limitador-54d6d988d5-tgmjb\" (UID: \"ffb6f688-9ecd-4c59-bdcc-707407d60eb2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tgmjb" Jul 10 18:26:53.562858 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.562840 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ffb6f688-9ecd-4c59-bdcc-707407d60eb2-config-file\") pod \"limitador-limitador-54d6d988d5-tgmjb\" (UID: \"ffb6f688-9ecd-4c59-bdcc-707407d60eb2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tgmjb" Jul 10 18:26:53.572911 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.572888 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74r99\" (UniqueName: \"kubernetes.io/projected/ffb6f688-9ecd-4c59-bdcc-707407d60eb2-kube-api-access-74r99\") pod \"limitador-limitador-54d6d988d5-tgmjb\" (UID: \"ffb6f688-9ecd-4c59-bdcc-707407d60eb2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tgmjb" Jul 10 18:26:53.670233 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.670141 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-tgmjb" Jul 10 18:26:53.810843 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.810819 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:26:53.813124 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:26:53.813080 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb6f688_9ecd_4c59_bdcc_707407d60eb2.slice/crio-c077e025f97c648e863f164a1ae5455abd9a04e2688849962ddd403e9ab27dd2 WatchSource:0}: Error finding container c077e025f97c648e863f164a1ae5455abd9a04e2688849962ddd403e9ab27dd2: Status 404 returned error can't find the container with id c077e025f97c648e863f164a1ae5455abd9a04e2688849962ddd403e9ab27dd2 Jul 10 18:26:53.844451 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.844417 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:53.946658 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:53.946631 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:26:54.035783 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:54.035750 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-tgmjb" event={"ID":"ffb6f688-9ecd-4c59-bdcc-707407d60eb2","Type":"ContainerStarted","Data":"c077e025f97c648e863f164a1ae5455abd9a04e2688849962ddd403e9ab27dd2"} Jul 10 18:26:57.051261 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:57.051220 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-tgmjb" event={"ID":"ffb6f688-9ecd-4c59-bdcc-707407d60eb2","Type":"ContainerStarted","Data":"26cecd47be9f651a7444ccd8b38f60f6d72d0eb5dbedb2b59d26d92c85a305f1"} Jul 10 18:26:57.051680 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:57.051415 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-tgmjb" Jul 10 18:26:57.068561 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:26:57.068512 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-tgmjb" podStartSLOduration=1.4033293279999999 podStartE2EDuration="4.068496847s" totalImagesPullingTime="2.665167519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:57.066796062 +0000 UTC m=+576.480080849" watchObservedRunningTime="2026-07-10 18:26:57.068496847 +0000 UTC m=+576.481781634" Jul 10 18:27:08.056644 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:08.056616 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-tgmjb" Jul 10 18:27:13.902319 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:13.902285 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:13.998185 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:13.998147 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:14.395222 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:14.395186 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:14.647070 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:14.646984 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:14.747159 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:14.747130 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:21.097041 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:21.097012 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:27:21.099755 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:21.099733 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:27:21.100771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:21.100751 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:27:21.103451 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:21.103435 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:27:26.191690 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:26.191653 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:27.392856 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:27.392813 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:29.503477 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:29.503424 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:29.559499 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:29.559466 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:32.801008 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:32.800980 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:32.893882 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:32.893849 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:32.959483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:32.959451 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:33.093546 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:33.093470 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:35.712746 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:35.712713 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:27:35.815470 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:27:35.815434 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:28:00.468789 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:00.468757 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:28:07.228252 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:07.228217 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:28:07.721106 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:07.721053 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:28:08.190241 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.190210 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:28:08.311062 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.311030 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:28:08.385255 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.385220 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-8b5nf"] Jul 10 18:28:08.388841 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.388821 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-8b5nf" Jul 10 18:28:08.392128 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.392105 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-5xfjs\"" Jul 10 18:28:08.392225 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.392183 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 10 18:28:08.405300 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.405272 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-8b5nf"] Jul 10 18:28:08.502090 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.502001 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:28:08.535170 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.535137 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj66l\" (UniqueName: \"kubernetes.io/projected/6ffce838-f116-412b-a24e-87fce044078e-kube-api-access-tj66l\") pod \"payload-pre-processing-7964457696-8b5nf\" (UID: \"6ffce838-f116-412b-a24e-87fce044078e\") " pod="openshift-ingress/payload-pre-processing-7964457696-8b5nf" Jul 10 18:28:08.535335 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.535268 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ffce838-f116-412b-a24e-87fce044078e-plugins-config-volume\") pod \"payload-pre-processing-7964457696-8b5nf\" (UID: \"6ffce838-f116-412b-a24e-87fce044078e\") " pod="openshift-ingress/payload-pre-processing-7964457696-8b5nf" Jul 10 18:28:08.636410 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.636357 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj66l\" (UniqueName: \"kubernetes.io/projected/6ffce838-f116-412b-a24e-87fce044078e-kube-api-access-tj66l\") pod \"payload-pre-processing-7964457696-8b5nf\" (UID: \"6ffce838-f116-412b-a24e-87fce044078e\") " pod="openshift-ingress/payload-pre-processing-7964457696-8b5nf" Jul 10 18:28:08.636611 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.636504 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ffce838-f116-412b-a24e-87fce044078e-plugins-config-volume\") pod \"payload-pre-processing-7964457696-8b5nf\" (UID: \"6ffce838-f116-412b-a24e-87fce044078e\") " pod="openshift-ingress/payload-pre-processing-7964457696-8b5nf" Jul 10 18:28:08.637219 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.637192 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ffce838-f116-412b-a24e-87fce044078e-plugins-config-volume\") pod \"payload-pre-processing-7964457696-8b5nf\" (UID: \"6ffce838-f116-412b-a24e-87fce044078e\") " pod="openshift-ingress/payload-pre-processing-7964457696-8b5nf" Jul 10 18:28:08.644678 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.644651 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj66l\" (UniqueName: \"kubernetes.io/projected/6ffce838-f116-412b-a24e-87fce044078e-kube-api-access-tj66l\") pod \"payload-pre-processing-7964457696-8b5nf\" (UID: \"6ffce838-f116-412b-a24e-87fce044078e\") " pod="openshift-ingress/payload-pre-processing-7964457696-8b5nf" Jul 10 18:28:08.699462 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.699431 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-8b5nf" Jul 10 18:28:08.836662 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:08.836601 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-8b5nf"] Jul 10 18:28:08.839406 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:28:08.839378 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ffce838_f116_412b_a24e_87fce044078e.slice/crio-3e8b8ad8abff334378945f56b8dd398b8b209b9bdc0a2c892ff29b0286613b6c WatchSource:0}: Error finding container 3e8b8ad8abff334378945f56b8dd398b8b209b9bdc0a2c892ff29b0286613b6c: Status 404 returned error can't find the container with id 3e8b8ad8abff334378945f56b8dd398b8b209b9bdc0a2c892ff29b0286613b6c Jul 10 18:28:09.382618 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:09.382577 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-8b5nf" event={"ID":"6ffce838-f116-412b-a24e-87fce044078e","Type":"ContainerStarted","Data":"3e8b8ad8abff334378945f56b8dd398b8b209b9bdc0a2c892ff29b0286613b6c"} Jul 10 18:28:10.388270 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:10.388231 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-8b5nf" event={"ID":"6ffce838-f116-412b-a24e-87fce044078e","Type":"ContainerStarted","Data":"391731530c83da0fd3b9c0b68e764853d117b57672569a5cd2f445606997b9f8"} Jul 10 18:28:10.388627 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:10.388359 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-8b5nf" Jul 10 18:28:10.404582 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:10.404535 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-8b5nf" podStartSLOduration=0.951930649 podStartE2EDuration="2.40451846s" totalImagesPullingTime="1.452587811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:28:10.403440068 +0000 UTC m=+649.816724865" watchObservedRunningTime="2026-07-10 18:28:10.40451846 +0000 UTC m=+649.817803225" Jul 10 18:28:18.636681 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:18.636649 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:28:18.726121 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:18.726067 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:28:21.395242 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:21.395210 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-8b5nf" Jul 10 18:28:31.129478 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:31.129433 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:28:31.661647 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:31.661610 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:28:34.904563 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:34.904529 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:28:35.006697 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:35.006661 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:28:35.107549 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:35.107517 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:28:35.973034 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:35.973000 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:28:46.306495 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:46.306461 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:28:47.476775 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:28:47.476739 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:29:00.477005 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.476969 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6"] Jul 10 18:29:00.480562 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.480539 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.483422 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.483401 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 18:29:00.484336 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.484315 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 18:29:00.484459 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.484378 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-m2g74\"" Jul 10 18:29:00.484459 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.484445 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 10 18:29:00.492574 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.492548 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6"] Jul 10 18:29:00.567972 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.567941 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:29:00.616912 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.616871 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.617125 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.616928 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.617125 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.616994 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.617125 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.617064 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.617125 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.617088 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.617292 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.617200 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h9fh\" (UniqueName: \"kubernetes.io/projected/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-kube-api-access-8h9fh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.717830 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.717788 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.717830 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.717836 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.718082 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.717854 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.718082 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.717897 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.718082 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.717918 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.718082 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.717969 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h9fh\" (UniqueName: \"kubernetes.io/projected/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-kube-api-access-8h9fh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.718334 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.718239 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.718334 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.718294 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.718407 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.718330 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.720197 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.720176 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.720459 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.720443 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.725584 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.725562 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h9fh\" (UniqueName: \"kubernetes.io/projected/8f5a9698-95df-4831-9c0e-daa4c7b29ce9-kube-api-access-8h9fh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6\" (UID: \"8f5a9698-95df-4831-9c0e-daa4c7b29ce9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.791947 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.791856 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:00.928267 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.928243 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6"] Jul 10 18:29:00.930312 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:29:00.930288 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f5a9698_95df_4831_9c0e_daa4c7b29ce9.slice/crio-cef46ce7b59b13c5c13384e9a95dab3490e913ad91e705ddbc8a8b2682e10ed7 WatchSource:0}: Error finding container cef46ce7b59b13c5c13384e9a95dab3490e913ad91e705ddbc8a8b2682e10ed7: Status 404 returned error can't find the container with id cef46ce7b59b13c5c13384e9a95dab3490e913ad91e705ddbc8a8b2682e10ed7 Jul 10 18:29:00.931700 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:00.931682 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:29:01.640056 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:01.640022 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" event={"ID":"8f5a9698-95df-4831-9c0e-daa4c7b29ce9","Type":"ContainerStarted","Data":"cef46ce7b59b13c5c13384e9a95dab3490e913ad91e705ddbc8a8b2682e10ed7"} Jul 10 18:29:02.376889 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:02.376783 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:29:02.646027 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:02.645923 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" event={"ID":"8f5a9698-95df-4831-9c0e-daa4c7b29ce9","Type":"ContainerStarted","Data":"b654e42e79d2b8c4951b0ba0c3f0e71ba22d3b35d1d7d3ccf20f9044face26ab"} Jul 10 18:29:08.441749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:08.441710 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:29:10.792960 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:10.792919 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:10.793389 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:10.793024 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:10.805302 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:10.805276 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:10.824311 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:10.824259 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" podStartSLOduration=9.768717612 podStartE2EDuration="10.824243158s" totalImagesPullingTime="1.055525546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:29:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:29:02.667747522 +0000 UTC m=+702.081032308" watchObservedRunningTime="2026-07-10 18:29:10.824243158 +0000 UTC m=+710.237527943" Jul 10 18:29:10.877323 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:10.877281 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:29:11.705746 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:11.705716 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6" Jul 10 18:29:14.952659 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:14.952577 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl"] Jul 10 18:29:14.956171 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:14.956147 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:14.958617 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:14.958602 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 10 18:29:14.967005 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:14.966977 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl"] Jul 10 18:29:14.983263 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:14.983231 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:29:15.044005 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.043963 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.044005 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.044006 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.044260 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.044027 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.044260 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.044052 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.044260 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.044152 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.044260 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.044189 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6jcg\" (UniqueName: \"kubernetes.io/projected/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-kube-api-access-b6jcg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.144680 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.144640 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.144680 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.144680 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.144909 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.144698 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.144909 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.144809 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.145471 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.145269 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.145471 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.145329 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.145471 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.145362 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6jcg\" (UniqueName: \"kubernetes.io/projected/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-kube-api-access-b6jcg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.145471 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.145368 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.145790 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.145679 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.152756 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.148062 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.154614 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.154583 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.154956 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.154937 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6jcg\" (UniqueName: \"kubernetes.io/projected/ed4743f8-b7fa-4e73-bd30-5fa474bef07c-kube-api-access-b6jcg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl\" (UID: \"ed4743f8-b7fa-4e73-bd30-5fa474bef07c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.265293 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.265204 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:15.402997 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.402962 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl"] Jul 10 18:29:15.405945 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:29:15.405896 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded4743f8_b7fa_4e73_bd30_5fa474bef07c.slice/crio-7d71ec2dffe0948585c91e9a68303cd906ea1257a181d394891fa294e3e5eeef WatchSource:0}: Error finding container 7d71ec2dffe0948585c91e9a68303cd906ea1257a181d394891fa294e3e5eeef: Status 404 returned error can't find the container with id 7d71ec2dffe0948585c91e9a68303cd906ea1257a181d394891fa294e3e5eeef Jul 10 18:29:15.716514 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.716477 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" event={"ID":"ed4743f8-b7fa-4e73-bd30-5fa474bef07c","Type":"ContainerStarted","Data":"6c1a7c96deebf9b921d381850cb1fe2397cc10241f800a8fd26d1afdfde5d1ce"} Jul 10 18:29:15.716514 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.716519 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" event={"ID":"ed4743f8-b7fa-4e73-bd30-5fa474bef07c","Type":"ContainerStarted","Data":"7d71ec2dffe0948585c91e9a68303cd906ea1257a181d394891fa294e3e5eeef"} Jul 10 18:29:15.737779 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:15.737717 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" podStartSLOduration=1.526551488 podStartE2EDuration="1.737700246s" totalImagesPullingTime="211.148758ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:29:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:29:15.736433392 +0000 UTC m=+715.149718182" watchObservedRunningTime="2026-07-10 18:29:15.737700246 +0000 UTC m=+715.150985034" Jul 10 18:29:18.281584 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:18.281550 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:29:19.944381 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:19.944349 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:29:20.044730 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:20.044700 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:29:24.934059 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:24.934011 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:29:25.265780 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:25.265685 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:25.265780 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:25.265734 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:25.278167 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:25.278141 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:29:25.778904 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:29:25.778871 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl" Jul 10 18:30:43.813766 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:30:43.813687 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:43.912603 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:30:43.912573 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:44.981123 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:30:44.981082 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:30:45.081354 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:30:45.081313 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:01.824643 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:31:01.824607 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:01.924980 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:31:01.924952 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:13.874043 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:31:13.874006 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:31:20.179373 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:31:20.179337 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:24.073465 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:31:24.073432 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:31:33.772758 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:31:33.772720 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:31:43.378761 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:31:43.376289 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:31:54.678575 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:31:54.678538 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:31:55.812799 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:31:55.812744 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:31:57.202970 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:31:57.202932 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:21.129408 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:21.129328 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:32:21.129974 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:21.129909 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:32:21.134171 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:21.134150 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:32:21.134390 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:21.134371 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:32:22.079339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:22.079295 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:32:38.379748 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:38.379707 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:32:43.619605 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:43.619532 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:45.358153 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:45.358124 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:49.622532 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:49.622498 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:49.684806 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:49.684776 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:49.771271 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:49.771237 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:49.869231 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:49.869193 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:49.982313 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:49.982275 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:55.726621 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:55.726565 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:32:55.790286 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:32:55.790252 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:33:14.568440 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:33:14.568399 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:33:22.573002 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:33:22.572956 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:33:33.478370 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:33:33.478331 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:33:41.171893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:33:41.171852 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:33:52.778270 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:33:52.778179 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:34:02.978959 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:34:02.978918 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:34:14.070811 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:34:14.070767 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:34:24.272956 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:34:24.272909 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:34:40.779748 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:34:40.779711 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:40.864233 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:34:40.864199 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:34:45.670644 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:34:45.670609 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:34:57.586694 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:34:57.586660 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:34:59.957480 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:34:59.957447 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:35:22.982085 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:35:22.981988 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:35:37.673410 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:35:37.673374 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:35:49.695786 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:35:49.695748 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:35:59.789599 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:35:59.789556 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:36:10.878232 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:36:10.878198 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:36:23.700796 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:36:23.700760 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:36:34.301381 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:36:34.301340 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:37:21.161327 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:37:21.161292 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:37:21.162561 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:37:21.162539 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:37:21.165595 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:37:21.165568 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:37:21.166752 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:37:21.166726 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:38:16.020079 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:38:16.019997 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:38:16.968625 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:38:16.968585 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:38:26.370743 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:38:26.370710 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:38:40.583964 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:38:40.583925 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:38:52.595736 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:38:52.595700 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:39:05.568686 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:39:05.568649 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:39:15.487395 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:39:15.487361 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:39:15.670874 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:39:15.670838 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:39:23.977447 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:39:23.977407 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:39:34.678170 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:39:34.678125 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:39:44.979299 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:39:44.979213 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:39:54.469183 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:39:54.469150 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:40:08.477433 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:40:08.477386 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:40:22.640960 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:40:22.640905 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:22.738647 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:40:22.738569 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:40:26.872650 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:40:26.872613 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:40:36.478021 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:40:36.477982 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:40:46.483730 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:40:46.483683 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:40:56.780644 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:40:56.780609 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:41:06.275703 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:41:06.275666 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:41:18.592856 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:41:18.592775 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:41:33.671967 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:41:33.671937 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:41:43.771496 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:41:43.771462 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:41:57.766116 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:41:57.766055 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:42:06.275976 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:42:06.275942 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:42:20.188411 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:42:20.188377 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:42:21.190884 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:42:21.190852 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:42:21.193383 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:42:21.193358 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:42:21.195370 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:42:21.195351 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:42:21.197209 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:42:21.197192 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:42:21.888366 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:42:21.888329 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:32.594988 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:42:32.594951 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:42:36.575268 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:42:36.575229 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:42:50.674511 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:42:50.674427 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:43:01.074045 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:43:01.074008 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:43:10.876498 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:43:10.876461 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:43:22.283767 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:43:22.283732 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:43:34.572497 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:43:34.572460 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:43:47.573039 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:43:47.572996 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:43:56.172304 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:43:56.172269 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:43:58.799536 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:43:58.799504 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:04.231330 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:04.231293 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:10.475651 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:10.475619 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:44:12.491862 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:12.491772 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:17.617811 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:17.617773 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:20.762908 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:20.762869 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:20.852499 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:20.852469 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:21.232608 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:21.232576 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:21.783281 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:21.783246 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:44:27.006364 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:27.006330 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:27.106415 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:27.106374 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:27.233857 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:27.233823 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:27.336083 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:27.335988 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:28.441175 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:28.441139 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:28.535203 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:28.535148 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:28.600605 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:28.600574 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:28.989573 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:28.989528 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:29.093710 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:29.093676 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:31.298774 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:31.298725 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:32.477940 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:32.477897 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:44:33.549698 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:33.549654 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:33.812068 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:33.811968 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:33.913647 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:33.913604 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:33.974415 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:33.974379 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:35.642712 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:35.642676 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:39.625662 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:39.625623 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:39.726584 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:39.726550 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:39.863734 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:39.863700 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:39.944901 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:39.944860 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:40.983223 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:40.983176 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:41.079717 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:41.079688 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:41.159540 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:41.159508 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:41.266320 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:41.266231 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:41.721045 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:41.721009 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:41.922889 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:41.922826 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:43.212614 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:43.212572 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:43.476760 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:43.476664 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:44:46.166826 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:46.166791 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:46.526380 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:46.526344 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:52.340250 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:52.340213 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:52.442674 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:52.442616 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:55.032665 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:55.032629 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:58.141983 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:58.141947 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:58.243440 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:58.243403 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:58.398470 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:58.398373 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:58.498888 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:58.498843 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:59.492521 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:59.492490 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:59.591521 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:59.591486 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:59.810194 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:59.810089 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:44:59.915077 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:44:59.915036 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:00.424166 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:00.424122 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:00.527625 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:00.527595 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:01.471587 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:01.471554 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:02.469988 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:02.469955 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:45:03.977603 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:03.977573 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:04.295663 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:04.295551 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:04.389859 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:04.389824 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:04.552301 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:04.552218 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:06.294566 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:06.294531 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:10.265469 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:10.265425 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:10.330415 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:10.330382 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:10.760400 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:10.760369 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:10.861646 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:10.861610 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:10.959362 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:10.959316 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:12.449334 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:12.449289 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:14.876626 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:14.876581 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:45:18.530059 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:18.530024 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:18.636981 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:18.636947 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:18.736350 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:18.736323 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:18.831249 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:18.831147 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:19.288187 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:19.288155 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:45.322512 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:45.322423 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:51.015684 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:51.015645 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:51.156280 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:51.156214 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:51.321323 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:51.321234 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:51.877359 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:51.877322 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:45:51.941150 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:45:51.941087 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:07.227663 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:07.227620 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:13.522675 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:13.522636 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:13.592584 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:13.592550 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:13.699873 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:13.699835 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:24.286720 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:24.286679 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:25.151415 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:25.151371 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:31.121564 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:31.121524 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:31.201063 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:31.201022 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:31.293385 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:31.293350 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:31.393242 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:31.393161 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:32.296964 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:32.296921 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:32.397991 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:32.397936 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:32.497371 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:32.497334 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:33.504684 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:33.504646 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:46:34.236464 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:34.236430 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:40.068911 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:40.068874 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:40.168682 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:40.168647 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:40.784803 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:40.784765 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:41.369141 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:41.369087 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:46:46.533314 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:46.533268 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:46.608663 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:46.608615 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:46.796428 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:46.796334 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:47.209798 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:47.209763 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:47.386243 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:47.386209 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:49.375870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:49.375840 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:46:50.553796 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:50.553758 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:50.644374 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:50.644341 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:50.829407 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:50.829317 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:51.265947 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:51.265906 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:51.379790 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:51.379720 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:51.478502 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:51.478468 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:55.073408 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:55.073371 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:55.787972 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:55.787937 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:55.881162 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:55.881127 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:46:56.244501 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:46:56.244461 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:00.863932 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:00.863894 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:01.014492 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:01.014454 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:01.175313 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:01.175281 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:47:02.205488 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:02.205448 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:02.306429 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:02.306394 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:02.638233 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:02.638135 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:03.025961 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:03.025932 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:03.126853 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:03.126818 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:08.834511 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:08.834477 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:08.901682 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:08.901650 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:09.150513 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:09.150425 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:09.379506 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:09.379470 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:09.532292 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:09.532243 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:09.681855 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:09.681820 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:09.969691 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:09.969657 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:13.189738 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:13.189640 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:47:15.812048 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:15.812013 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:15.919453 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:15.919417 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:16.015239 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:16.015207 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:16.178325 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:16.178296 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:19.758553 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:19.758521 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:19.861216 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:19.861177 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:21.221592 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:21.221565 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:47:21.223645 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:21.223621 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:47:21.225945 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:21.225915 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:47:21.227431 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:21.227411 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:47:21.924428 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:21.924389 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:21.987488 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:21.987449 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:28.916195 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:28.916107 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:29.253714 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:29.253676 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:29.706788 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:29.706755 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:32.733449 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:32.733414 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:32.867544 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:32.867507 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:33.024870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:33.024778 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:33.210446 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:33.210415 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:33.526457 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:33.526425 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:33.628427 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:33.628393 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:37.257012 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:37.256979 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:39.777165 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:39.777128 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:47:48.298715 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.298676 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s"] Jul 10 18:47:48.302358 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.302334 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.306978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.306951 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-0a09b19d\"/\"e2e-lifecycle-model-0a09b19d-kserve-self-signed-certs\"" Jul 10 18:47:48.307132 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.307004 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-0a09b19d\"/\"openshift-service-ca.crt\"" Jul 10 18:47:48.307132 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.306951 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-0a09b19d\"/\"default-dockercfg-dhdw5\"" Jul 10 18:47:48.307210 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.306951 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-0a09b19d\"/\"kube-root-ca.crt\"" Jul 10 18:47:48.324727 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.324691 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s"] Jul 10 18:47:48.429630 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.429593 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:47:48.431798 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.431778 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-dshm\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.431885 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.431817 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6714b445-d607-4165-9be0-16480e70491c-tls-certs\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.431885 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.431863 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-home\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.431966 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.431889 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-model-cache\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.432001 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.431966 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvkdj\" (UniqueName: \"kubernetes.io/projected/6714b445-d607-4165-9be0-16480e70491c-kube-api-access-wvkdj\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.432035 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.431997 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-tmp-dir\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.533056 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.533020 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-dshm\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.533241 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.533073 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6714b445-d607-4165-9be0-16480e70491c-tls-certs\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.533288 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.533235 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-home\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.533288 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.533283 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-model-cache\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.533352 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.533336 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvkdj\" (UniqueName: \"kubernetes.io/projected/6714b445-d607-4165-9be0-16480e70491c-kube-api-access-wvkdj\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.533406 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.533369 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-tmp-dir\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.533598 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.533564 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-home\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.533691 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.533647 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-model-cache\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.533740 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.533698 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-tmp-dir\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.535401 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.535383 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-dshm\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.535585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.535561 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6714b445-d607-4165-9be0-16480e70491c-tls-certs\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.543829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.543796 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvkdj\" (UniqueName: \"kubernetes.io/projected/6714b445-d607-4165-9be0-16480e70491c-kube-api-access-wvkdj\") pod \"e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.612337 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.612245 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:48.759515 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:48.759488 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s"] Jul 10 18:47:48.761744 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:47:48.761710 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6714b445_d607_4165_9be0_16480e70491c.slice/crio-724ef477b01877de62890c1ed10b7eb8d60a9182097cddd2cfe24e97f9f2b692 WatchSource:0}: Error finding container 724ef477b01877de62890c1ed10b7eb8d60a9182097cddd2cfe24e97f9f2b692: Status 404 returned error can't find the container with id 724ef477b01877de62890c1ed10b7eb8d60a9182097cddd2cfe24e97f9f2b692 Jul 10 18:47:49.100526 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:49.100489 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" event={"ID":"6714b445-d607-4165-9be0-16480e70491c","Type":"ContainerStarted","Data":"98c31461aae1861163870078acc00259d66e881a81e5c2ad5a3ce11d1c70ae81"} Jul 10 18:47:49.100526 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:49.100530 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" event={"ID":"6714b445-d607-4165-9be0-16480e70491c","Type":"ContainerStarted","Data":"724ef477b01877de62890c1ed10b7eb8d60a9182097cddd2cfe24e97f9f2b692"} Jul 10 18:47:49.120211 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:49.120156 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" podStartSLOduration=1.120138556 podStartE2EDuration="1.120138556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:47:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:47:49.118855222 +0000 UTC m=+1828.532140011" watchObservedRunningTime="2026-07-10 18:47:49.120138556 +0000 UTC m=+1828.533423372" Jul 10 18:47:55.274061 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:55.274015 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:47:58.612673 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:58.612641 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:58.613107 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:58.612702 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:58.624582 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:58.624555 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:47:59.160668 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:47:59.160640 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:48:12.982985 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:12.982948 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:48:15.063982 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:15.063941 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:48:15.569933 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:15.569901 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:48:21.183298 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:21.183198 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:48:26.158918 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:26.158881 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s"] Jul 10 18:48:26.204053 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:26.204014 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:48:26.572325 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:26.572292 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:48:41.179889 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.179844 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" podUID="6714b445-d607-4165-9be0-16480e70491c" containerName="main" containerID="cri-o://98c31461aae1861163870078acc00259d66e881a81e5c2ad5a3ce11d1c70ae81" gracePeriod=45 Jul 10 18:48:41.352952 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.352920 2574 generic.go:356] "Generic (PLEG): container finished" podID="6714b445-d607-4165-9be0-16480e70491c" containerID="98c31461aae1861163870078acc00259d66e881a81e5c2ad5a3ce11d1c70ae81" exitCode=0 Jul 10 18:48:41.353177 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.353004 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" event={"ID":"6714b445-d607-4165-9be0-16480e70491c","Type":"ContainerDied","Data":"98c31461aae1861163870078acc00259d66e881a81e5c2ad5a3ce11d1c70ae81"} Jul 10 18:48:41.439888 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.439823 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:48:41.531275 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.531245 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6714b445-d607-4165-9be0-16480e70491c-tls-certs\") pod \"6714b445-d607-4165-9be0-16480e70491c\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " Jul 10 18:48:41.531465 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.531297 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-model-cache\") pod \"6714b445-d607-4165-9be0-16480e70491c\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " Jul 10 18:48:41.531465 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.531350 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvkdj\" (UniqueName: \"kubernetes.io/projected/6714b445-d607-4165-9be0-16480e70491c-kube-api-access-wvkdj\") pod \"6714b445-d607-4165-9be0-16480e70491c\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " Jul 10 18:48:41.531465 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.531448 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-dshm\") pod \"6714b445-d607-4165-9be0-16480e70491c\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " Jul 10 18:48:41.531631 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.531471 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-home\") pod \"6714b445-d607-4165-9be0-16480e70491c\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " Jul 10 18:48:41.531631 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.531498 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-tmp-dir\") pod \"6714b445-d607-4165-9be0-16480e70491c\" (UID: \"6714b445-d607-4165-9be0-16480e70491c\") " Jul 10 18:48:41.531631 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.531552 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-model-cache" (OuterVolumeSpecName: "model-cache") pod "6714b445-d607-4165-9be0-16480e70491c" (UID: "6714b445-d607-4165-9be0-16480e70491c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:48:41.531830 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.531776 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6714b445-d607-4165-9be0-16480e70491c" (UID: "6714b445-d607-4165-9be0-16480e70491c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:48:41.531830 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.531788 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-home" (OuterVolumeSpecName: "home") pod "6714b445-d607-4165-9be0-16480e70491c" (UID: "6714b445-d607-4165-9be0-16480e70491c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:48:41.531830 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.531806 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-model-cache\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:48:41.533733 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.533709 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-dshm" (OuterVolumeSpecName: "dshm") pod "6714b445-d607-4165-9be0-16480e70491c" (UID: "6714b445-d607-4165-9be0-16480e70491c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:48:41.533848 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.533741 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6714b445-d607-4165-9be0-16480e70491c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6714b445-d607-4165-9be0-16480e70491c" (UID: "6714b445-d607-4165-9be0-16480e70491c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:48:41.533848 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.533833 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6714b445-d607-4165-9be0-16480e70491c-kube-api-access-wvkdj" (OuterVolumeSpecName: "kube-api-access-wvkdj") pod "6714b445-d607-4165-9be0-16480e70491c" (UID: "6714b445-d607-4165-9be0-16480e70491c"). InnerVolumeSpecName "kube-api-access-wvkdj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:48:41.632565 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.632528 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-dshm\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:48:41.632565 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.632558 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-home\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:48:41.632565 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.632566 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6714b445-d607-4165-9be0-16480e70491c-tmp-dir\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:48:41.632801 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.632575 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6714b445-d607-4165-9be0-16480e70491c-tls-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:48:41.632801 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:41.632587 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvkdj\" (UniqueName: \"kubernetes.io/projected/6714b445-d607-4165-9be0-16480e70491c-kube-api-access-wvkdj\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:48:42.360041 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:42.359971 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" Jul 10 18:48:42.360041 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:42.360014 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s" event={"ID":"6714b445-d607-4165-9be0-16480e70491c","Type":"ContainerDied","Data":"724ef477b01877de62890c1ed10b7eb8d60a9182097cddd2cfe24e97f9f2b692"} Jul 10 18:48:42.360502 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:42.360065 2574 scope.go:117] "RemoveContainer" containerID="98c31461aae1861163870078acc00259d66e881a81e5c2ad5a3ce11d1c70ae81" Jul 10 18:48:42.383374 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:42.383342 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s"] Jul 10 18:48:42.389701 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:42.389672 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-0a09b19d/e2e-lifecycle-model-0a09b19d-kserve-869c4d947f-b7p6s"] Jul 10 18:48:43.321156 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:43.317885 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6714b445-d607-4165-9be0-16480e70491c" path="/var/lib/kubelet/pods/6714b445-d607-4165-9be0-16480e70491c/volumes" Jul 10 18:48:44.914385 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:48:44.914346 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:02.856381 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:02.856332 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:04.162372 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:04.162338 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:06.167729 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:06.167688 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:49:08.949577 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:08.949532 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:09.049561 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:09.049529 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:09.335016 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:09.334925 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:09.513575 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:09.513533 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:10.575226 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:10.575189 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:10.677254 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:10.677201 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:10.775869 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:10.775832 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:11.639727 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:11.639687 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:11.729963 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:11.729927 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:13.166685 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:13.166650 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:17.380448 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:17.380408 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:49:19.015835 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:19.015796 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:19.117350 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:19.117319 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:19.717437 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:19.717400 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:25.532323 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:25.532284 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:25.642460 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:25.642419 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:25.776263 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:25.776220 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:27.179676 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:27.179640 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:49:36.336776 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:36.336736 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:36.626833 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:36.626744 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:49:57.637903 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:49:57.637861 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:03.494248 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:03.494201 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:03.593891 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:03.593851 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:04.025628 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:04.025540 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:04.312837 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:04.312713 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:04.765168 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:04.765127 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:07.582219 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:07.582176 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:07.689335 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:07.689284 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:07.813281 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:07.813243 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:07.903491 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:07.903407 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:08.273138 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:08.273070 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:08.374415 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:08.374378 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:12.104008 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:12.103973 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:12.508410 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:12.508374 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:12.881634 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:12.881537 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:13.154451 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:13.154368 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:15.529204 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:15.529164 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:15.625674 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:15.625637 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:15.744147 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:15.744087 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:15.937078 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:15.937040 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:15.992632 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:15.992591 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:50:16.351392 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:16.351301 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:16.451636 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:16.451605 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:20.033084 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:20.033049 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:29.243486 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:29.243442 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:29.345640 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:29.345605 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:30.477830 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:30.477794 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:50:37.069163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:37.069122 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:37.169650 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:37.169619 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:47.771894 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.771855 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k"] Jul 10 18:50:47.772532 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.772424 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6714b445-d607-4165-9be0-16480e70491c" containerName="main" Jul 10 18:50:47.772532 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.772445 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6714b445-d607-4165-9be0-16480e70491c" containerName="main" Jul 10 18:50:47.772652 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.772589 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6714b445-d607-4165-9be0-16480e70491c" containerName="main" Jul 10 18:50:47.775924 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.775900 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:47.779623 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.779597 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-bee2bc\"/\"test-model-bee2bc-kserve-self-signed-certs\"" Jul 10 18:50:47.779789 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.779597 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-bee2bc\"/\"default-dockercfg-tllt8\"" Jul 10 18:50:47.779789 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.779656 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-bee2bc\"/\"openshift-service-ca.crt\"" Jul 10 18:50:47.779789 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.779704 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-bee2bc\"/\"kube-root-ca.crt\"" Jul 10 18:50:47.786788 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.786764 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k"] Jul 10 18:50:47.829887 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.829854 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:50:47.905200 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.905163 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e42f61d-30d8-4887-9bf3-d00526715a68-tls-certs\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:47.905416 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.905275 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-home\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:47.905416 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.905299 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-model-cache\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:47.905416 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.905338 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-dshm\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:47.905416 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.905366 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-tmp-dir\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:47.905624 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:47.905507 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9smr\" (UniqueName: \"kubernetes.io/projected/7e42f61d-30d8-4887-9bf3-d00526715a68-kube-api-access-t9smr\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:48.006773 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.006731 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9smr\" (UniqueName: \"kubernetes.io/projected/7e42f61d-30d8-4887-9bf3-d00526715a68-kube-api-access-t9smr\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:48.006773 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.006777 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e42f61d-30d8-4887-9bf3-d00526715a68-tls-certs\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:48.007030 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.006823 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-home\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:48.007030 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.006839 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-model-cache\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:48.007030 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.006861 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-dshm\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:48.007030 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.006879 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-tmp-dir\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:48.007331 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.007305 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-home\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:48.007382 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.007347 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-model-cache\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:48.007416 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.007385 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-tmp-dir\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:48.009279 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.009259 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-dshm\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:48.009448 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.009432 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e42f61d-30d8-4887-9bf3-d00526715a68-tls-certs\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:48.016910 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.016883 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9smr\" (UniqueName: \"kubernetes.io/projected/7e42f61d-30d8-4887-9bf3-d00526715a68-kube-api-access-t9smr\") pod \"test-model-bee2bc-kserve-76b5669579-92n8k\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:48.086207 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.086114 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:48.223899 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.223873 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k"] Jul 10 18:50:48.226577 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:50:48.226551 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e42f61d_30d8_4887_9bf3_d00526715a68.slice/crio-e54e956ddfbb6ee5c1685ae5dbb30ce3552e9d947ca139eeb6bbddedb9b84558 WatchSource:0}: Error finding container e54e956ddfbb6ee5c1685ae5dbb30ce3552e9d947ca139eeb6bbddedb9b84558: Status 404 returned error can't find the container with id e54e956ddfbb6ee5c1685ae5dbb30ce3552e9d947ca139eeb6bbddedb9b84558 Jul 10 18:50:48.976869 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.976834 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" event={"ID":"7e42f61d-30d8-4887-9bf3-d00526715a68","Type":"ContainerStarted","Data":"7a6c3d62c66e31ade26546f4108c190248a7505ad502a6e17bb0902206376fd6"} Jul 10 18:50:48.977262 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.976882 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" event={"ID":"7e42f61d-30d8-4887-9bf3-d00526715a68","Type":"ContainerStarted","Data":"e54e956ddfbb6ee5c1685ae5dbb30ce3552e9d947ca139eeb6bbddedb9b84558"} Jul 10 18:50:48.995370 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:48.995309 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" podStartSLOduration=1.995288354 podStartE2EDuration="1.995288354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:50:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:50:48.99341746 +0000 UTC m=+2008.406702249" watchObservedRunningTime="2026-07-10 18:50:48.995288354 +0000 UTC m=+2008.408573142" Jul 10 18:50:51.484620 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:51.484588 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:50:58.087232 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:58.087192 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:58.087729 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:58.087447 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:58.099783 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:58.099756 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:59.037648 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:59.037619 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:50:59.822617 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:50:59.822579 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:51:03.179180 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:03.179145 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:51:24.480962 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.480924 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56"] Jul 10 18:51:24.484748 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.484723 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.487424 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.487403 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9a1ca1\"/\"openshift-service-ca.crt\"" Jul 10 18:51:24.487424 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.487418 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9a1ca1\"/\"kube-root-ca.crt\"" Jul 10 18:51:24.488369 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.488350 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9a1ca1\"/\"default-dockercfg-wkb8b\"" Jul 10 18:51:24.488483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.488388 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9a1ca1\"/\"test-model-9a1ca1-kserve-self-signed-certs\"" Jul 10 18:51:24.493465 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.493441 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56"] Jul 10 18:51:24.504028 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.503994 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:51:24.548879 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.548842 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-dshm\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.549058 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.548891 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-tmp-dir\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.549058 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.548912 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-home\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.549058 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.548994 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8af1afdc-5110-4b63-bb48-aadb74bdf92b-tls-certs\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.549058 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.549016 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-model-cache\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.549221 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.549125 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9wpk\" (UniqueName: \"kubernetes.io/projected/8af1afdc-5110-4b63-bb48-aadb74bdf92b-kube-api-access-d9wpk\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.649923 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.649885 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9wpk\" (UniqueName: \"kubernetes.io/projected/8af1afdc-5110-4b63-bb48-aadb74bdf92b-kube-api-access-d9wpk\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.649923 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.649931 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-dshm\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.650224 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.649965 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-tmp-dir\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.650224 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.649990 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-home\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.650224 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.650049 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8af1afdc-5110-4b63-bb48-aadb74bdf92b-tls-certs\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.650224 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.650078 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-model-cache\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.650514 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.650447 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-tmp-dir\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.650567 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.650505 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-home\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.650567 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.650536 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-model-cache\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.652224 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.652205 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-dshm\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.652472 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.652453 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8af1afdc-5110-4b63-bb48-aadb74bdf92b-tls-certs\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.658846 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.658820 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9wpk\" (UniqueName: \"kubernetes.io/projected/8af1afdc-5110-4b63-bb48-aadb74bdf92b-kube-api-access-d9wpk\") pod \"test-model-9a1ca1-kserve-7b585c679c-mvq56\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.795338 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.795247 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:24.936186 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:24.936160 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56"] Jul 10 18:51:24.938489 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:51:24.938460 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8af1afdc_5110_4b63_bb48_aadb74bdf92b.slice/crio-c71c1f6d852a9553556fe1faa3bcb75035b070a683fa1e75ef8f4393ffc1fe6f WatchSource:0}: Error finding container c71c1f6d852a9553556fe1faa3bcb75035b070a683fa1e75ef8f4393ffc1fe6f: Status 404 returned error can't find the container with id c71c1f6d852a9553556fe1faa3bcb75035b070a683fa1e75ef8f4393ffc1fe6f Jul 10 18:51:25.166465 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:25.166381 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" event={"ID":"8af1afdc-5110-4b63-bb48-aadb74bdf92b","Type":"ContainerStarted","Data":"1478ff1b736cf6529655045d1044f561085a7de9b8b326d4d995c55780b6f4d0"} Jul 10 18:51:25.166465 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:25.166421 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" event={"ID":"8af1afdc-5110-4b63-bb48-aadb74bdf92b","Type":"ContainerStarted","Data":"c71c1f6d852a9553556fe1faa3bcb75035b070a683fa1e75ef8f4393ffc1fe6f"} Jul 10 18:51:25.184178 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:25.184133 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" podStartSLOduration=1.184114818 podStartE2EDuration="1.184114818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:51:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:51:25.183080797 +0000 UTC m=+2044.596365586" watchObservedRunningTime="2026-07-10 18:51:25.184114818 +0000 UTC m=+2044.597399602" Jul 10 18:51:28.882040 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:28.881997 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:51:34.795583 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:34.795539 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:34.796084 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:34.795631 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:34.808236 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:34.808210 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:35.232516 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:35.232490 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:51:42.988933 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:42.988845 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:51:45.006702 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:45.006665 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k"] Jul 10 18:51:45.027926 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:45.027886 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:51:46.018776 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:46.018734 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56"] Jul 10 18:51:46.040556 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:46.040531 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:51:46.198313 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:46.198276 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:51:46.642543 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:46.642508 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:51:46.741475 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:46.741438 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:51:46.928965 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:46.928908 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:51:51.705227 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:51.705186 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:51:58.874350 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:51:58.874308 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:52:00.027411 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.027353 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" podUID="7e42f61d-30d8-4887-9bf3-d00526715a68" containerName="main" containerID="cri-o://7a6c3d62c66e31ade26546f4108c190248a7505ad502a6e17bb0902206376fd6" gracePeriod=45 Jul 10 18:52:00.270260 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.270236 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:52:00.294679 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.294608 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-dshm\") pod \"7e42f61d-30d8-4887-9bf3-d00526715a68\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " Jul 10 18:52:00.294679 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.294656 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9smr\" (UniqueName: \"kubernetes.io/projected/7e42f61d-30d8-4887-9bf3-d00526715a68-kube-api-access-t9smr\") pod \"7e42f61d-30d8-4887-9bf3-d00526715a68\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " Jul 10 18:52:00.294877 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.294737 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-tmp-dir\") pod \"7e42f61d-30d8-4887-9bf3-d00526715a68\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " Jul 10 18:52:00.294877 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.294759 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e42f61d-30d8-4887-9bf3-d00526715a68-tls-certs\") pod \"7e42f61d-30d8-4887-9bf3-d00526715a68\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " Jul 10 18:52:00.294877 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.294781 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-model-cache\") pod \"7e42f61d-30d8-4887-9bf3-d00526715a68\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " Jul 10 18:52:00.294877 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.294804 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-home\") pod \"7e42f61d-30d8-4887-9bf3-d00526715a68\" (UID: \"7e42f61d-30d8-4887-9bf3-d00526715a68\") " Jul 10 18:52:00.295085 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.295029 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e42f61d-30d8-4887-9bf3-d00526715a68" (UID: "7e42f61d-30d8-4887-9bf3-d00526715a68"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:00.295246 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.295217 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e42f61d-30d8-4887-9bf3-d00526715a68" (UID: "7e42f61d-30d8-4887-9bf3-d00526715a68"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:00.295463 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.295312 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-home" (OuterVolumeSpecName: "home") pod "7e42f61d-30d8-4887-9bf3-d00526715a68" (UID: "7e42f61d-30d8-4887-9bf3-d00526715a68"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:00.295463 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.295361 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-tmp-dir\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:52:00.297248 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.297202 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e42f61d-30d8-4887-9bf3-d00526715a68-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e42f61d-30d8-4887-9bf3-d00526715a68" (UID: "7e42f61d-30d8-4887-9bf3-d00526715a68"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:52:00.297499 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.297480 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-dshm" (OuterVolumeSpecName: "dshm") pod "7e42f61d-30d8-4887-9bf3-d00526715a68" (UID: "7e42f61d-30d8-4887-9bf3-d00526715a68"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:00.297560 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.297541 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e42f61d-30d8-4887-9bf3-d00526715a68-kube-api-access-t9smr" (OuterVolumeSpecName: "kube-api-access-t9smr") pod "7e42f61d-30d8-4887-9bf3-d00526715a68" (UID: "7e42f61d-30d8-4887-9bf3-d00526715a68"). InnerVolumeSpecName "kube-api-access-t9smr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:52:00.351729 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.351692 2574 generic.go:356] "Generic (PLEG): container finished" podID="7e42f61d-30d8-4887-9bf3-d00526715a68" containerID="7a6c3d62c66e31ade26546f4108c190248a7505ad502a6e17bb0902206376fd6" exitCode=0 Jul 10 18:52:00.351903 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.351738 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" Jul 10 18:52:00.351903 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.351779 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" event={"ID":"7e42f61d-30d8-4887-9bf3-d00526715a68","Type":"ContainerDied","Data":"7a6c3d62c66e31ade26546f4108c190248a7505ad502a6e17bb0902206376fd6"} Jul 10 18:52:00.351903 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.351826 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k" event={"ID":"7e42f61d-30d8-4887-9bf3-d00526715a68","Type":"ContainerDied","Data":"e54e956ddfbb6ee5c1685ae5dbb30ce3552e9d947ca139eeb6bbddedb9b84558"} Jul 10 18:52:00.351903 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.351849 2574 scope.go:117] "RemoveContainer" containerID="7a6c3d62c66e31ade26546f4108c190248a7505ad502a6e17bb0902206376fd6" Jul 10 18:52:00.361126 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.361109 2574 scope.go:117] "RemoveContainer" containerID="7a6c3d62c66e31ade26546f4108c190248a7505ad502a6e17bb0902206376fd6" Jul 10 18:52:00.361388 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:52:00.361369 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a6c3d62c66e31ade26546f4108c190248a7505ad502a6e17bb0902206376fd6\": container with ID starting with 7a6c3d62c66e31ade26546f4108c190248a7505ad502a6e17bb0902206376fd6 not found: ID does not exist" containerID="7a6c3d62c66e31ade26546f4108c190248a7505ad502a6e17bb0902206376fd6" Jul 10 18:52:00.361455 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.361402 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a6c3d62c66e31ade26546f4108c190248a7505ad502a6e17bb0902206376fd6"} err="failed to get container status \"7a6c3d62c66e31ade26546f4108c190248a7505ad502a6e17bb0902206376fd6\": rpc error: code = NotFound desc = could not find container \"7a6c3d62c66e31ade26546f4108c190248a7505ad502a6e17bb0902206376fd6\": container with ID starting with 7a6c3d62c66e31ade26546f4108c190248a7505ad502a6e17bb0902206376fd6 not found: ID does not exist" Jul 10 18:52:00.373349 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.373323 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k"] Jul 10 18:52:00.376641 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.376615 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-bee2bc/test-model-bee2bc-kserve-76b5669579-92n8k"] Jul 10 18:52:00.396352 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.396327 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-model-cache\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:52:00.396352 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.396351 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-home\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:52:00.396484 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.396358 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e42f61d-30d8-4887-9bf3-d00526715a68-dshm\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:52:00.396484 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.396367 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9smr\" (UniqueName: \"kubernetes.io/projected/7e42f61d-30d8-4887-9bf3-d00526715a68-kube-api-access-t9smr\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:52:00.396484 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:00.396377 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e42f61d-30d8-4887-9bf3-d00526715a68-tls-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:52:01.039916 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.039878 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" podUID="8af1afdc-5110-4b63-bb48-aadb74bdf92b" containerName="main" containerID="cri-o://1478ff1b736cf6529655045d1044f561085a7de9b8b326d4d995c55780b6f4d0" gracePeriod=45 Jul 10 18:52:01.295676 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.295612 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:52:01.314910 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.314873 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e42f61d-30d8-4887-9bf3-d00526715a68" path="/var/lib/kubelet/pods/7e42f61d-30d8-4887-9bf3-d00526715a68/volumes" Jul 10 18:52:01.358075 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.358040 2574 generic.go:356] "Generic (PLEG): container finished" podID="8af1afdc-5110-4b63-bb48-aadb74bdf92b" containerID="1478ff1b736cf6529655045d1044f561085a7de9b8b326d4d995c55780b6f4d0" exitCode=0 Jul 10 18:52:01.358259 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.358088 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" Jul 10 18:52:01.358259 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.358123 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" event={"ID":"8af1afdc-5110-4b63-bb48-aadb74bdf92b","Type":"ContainerDied","Data":"1478ff1b736cf6529655045d1044f561085a7de9b8b326d4d995c55780b6f4d0"} Jul 10 18:52:01.358259 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.358161 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56" event={"ID":"8af1afdc-5110-4b63-bb48-aadb74bdf92b","Type":"ContainerDied","Data":"c71c1f6d852a9553556fe1faa3bcb75035b070a683fa1e75ef8f4393ffc1fe6f"} Jul 10 18:52:01.358259 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.358181 2574 scope.go:117] "RemoveContainer" containerID="1478ff1b736cf6529655045d1044f561085a7de9b8b326d4d995c55780b6f4d0" Jul 10 18:52:01.366674 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.366653 2574 scope.go:117] "RemoveContainer" containerID="1478ff1b736cf6529655045d1044f561085a7de9b8b326d4d995c55780b6f4d0" Jul 10 18:52:01.366929 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:52:01.366912 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1478ff1b736cf6529655045d1044f561085a7de9b8b326d4d995c55780b6f4d0\": container with ID starting with 1478ff1b736cf6529655045d1044f561085a7de9b8b326d4d995c55780b6f4d0 not found: ID does not exist" containerID="1478ff1b736cf6529655045d1044f561085a7de9b8b326d4d995c55780b6f4d0" Jul 10 18:52:01.366979 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.366944 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1478ff1b736cf6529655045d1044f561085a7de9b8b326d4d995c55780b6f4d0"} err="failed to get container status \"1478ff1b736cf6529655045d1044f561085a7de9b8b326d4d995c55780b6f4d0\": rpc error: code = NotFound desc = could not find container \"1478ff1b736cf6529655045d1044f561085a7de9b8b326d4d995c55780b6f4d0\": container with ID starting with 1478ff1b736cf6529655045d1044f561085a7de9b8b326d4d995c55780b6f4d0 not found: ID does not exist" Jul 10 18:52:01.406823 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.406795 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-tmp-dir\") pod \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " Jul 10 18:52:01.406981 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.406897 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8af1afdc-5110-4b63-bb48-aadb74bdf92b-tls-certs\") pod \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " Jul 10 18:52:01.406981 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.406934 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9wpk\" (UniqueName: \"kubernetes.io/projected/8af1afdc-5110-4b63-bb48-aadb74bdf92b-kube-api-access-d9wpk\") pod \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " Jul 10 18:52:01.406981 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.406950 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-home\") pod \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " Jul 10 18:52:01.407163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.406986 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-dshm\") pod \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " Jul 10 18:52:01.407163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.407038 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-model-cache\") pod \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\" (UID: \"8af1afdc-5110-4b63-bb48-aadb74bdf92b\") " Jul 10 18:52:01.407163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.407090 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8af1afdc-5110-4b63-bb48-aadb74bdf92b" (UID: "8af1afdc-5110-4b63-bb48-aadb74bdf92b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:01.407368 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.407332 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-model-cache" (OuterVolumeSpecName: "model-cache") pod "8af1afdc-5110-4b63-bb48-aadb74bdf92b" (UID: "8af1afdc-5110-4b63-bb48-aadb74bdf92b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:01.407368 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.407347 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-home" (OuterVolumeSpecName: "home") pod "8af1afdc-5110-4b63-bb48-aadb74bdf92b" (UID: "8af1afdc-5110-4b63-bb48-aadb74bdf92b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:01.407488 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.407394 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-tmp-dir\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:52:01.409279 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.409248 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-dshm" (OuterVolumeSpecName: "dshm") pod "8af1afdc-5110-4b63-bb48-aadb74bdf92b" (UID: "8af1afdc-5110-4b63-bb48-aadb74bdf92b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:01.409394 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.409291 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8af1afdc-5110-4b63-bb48-aadb74bdf92b-kube-api-access-d9wpk" (OuterVolumeSpecName: "kube-api-access-d9wpk") pod "8af1afdc-5110-4b63-bb48-aadb74bdf92b" (UID: "8af1afdc-5110-4b63-bb48-aadb74bdf92b"). InnerVolumeSpecName "kube-api-access-d9wpk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:52:01.409394 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.409290 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8af1afdc-5110-4b63-bb48-aadb74bdf92b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8af1afdc-5110-4b63-bb48-aadb74bdf92b" (UID: "8af1afdc-5110-4b63-bb48-aadb74bdf92b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:52:01.508412 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.508379 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8af1afdc-5110-4b63-bb48-aadb74bdf92b-tls-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:52:01.508412 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.508407 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9wpk\" (UniqueName: \"kubernetes.io/projected/8af1afdc-5110-4b63-bb48-aadb74bdf92b-kube-api-access-d9wpk\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:52:01.508412 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.508416 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-home\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:52:01.508644 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.508424 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-dshm\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:52:01.508644 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.508432 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8af1afdc-5110-4b63-bb48-aadb74bdf92b-model-cache\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:52:01.683565 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.683535 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56"] Jul 10 18:52:01.690692 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:01.690668 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-9a1ca1/test-model-9a1ca1-kserve-7b585c679c-mvq56"] Jul 10 18:52:03.315191 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:03.315154 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8af1afdc-5110-4b63-bb48-aadb74bdf92b" path="/var/lib/kubelet/pods/8af1afdc-5110-4b63-bb48-aadb74bdf92b/volumes" Jul 10 18:52:20.729613 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:20.729577 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:20.878085 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:20.878030 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:21.250522 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:21.250492 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:52:21.254163 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:21.254142 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:52:21.254805 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:21.254783 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:52:21.258071 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:21.258048 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:52:28.571717 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:28.571677 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:28.859623 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:28.859537 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:29.098603 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:29.098569 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:29.266915 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:29.266878 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:29.695078 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:29.695040 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:29.883053 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:29.883014 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:29.983276 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:29.983157 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:30.166531 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:30.166502 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:33.186889 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:33.186851 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:52:35.410624 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:35.410584 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:35.511595 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:35.511540 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:35.782478 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:35.782447 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:36.002460 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:36.002428 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:36.365744 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:36.365709 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:36.748924 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:36.748890 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:36.897409 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:36.897379 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:39.378406 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:39.378367 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:39.465402 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:39.465367 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:39.541682 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:39.541645 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:39.992946 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:39.992912 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:40.093516 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:40.093479 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:40.194820 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:40.194770 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:43.915892 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:43.915858 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:44.417775 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:44.417730 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:44.743158 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:44.743118 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:45.040415 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:45.040329 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:47.484313 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:47.484261 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:47.597011 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:47.596976 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:52:47.605844 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:47.605803 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:47.940933 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:47.940881 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:48.113022 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:48.112984 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:48.765551 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:48.765514 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:48.873202 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:48.873168 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:52:51.974112 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:52:51.974050 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:00.428536 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:00.428504 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:00.772060 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:00.771966 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:00.930153 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:00.930121 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:01.029538 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:01.029448 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:01.253482 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:01.253448 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:05.174984 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:05.174952 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:53:07.188789 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:07.188754 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:07.291581 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:07.291546 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:07.544394 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:07.544301 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:07.812396 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:07.812296 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:07.946013 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:07.945946 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:08.374719 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:08.374690 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:08.487978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:08.487942 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:08.587051 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:08.587021 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:08.802518 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:08.802477 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:14.786247 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:14.786158 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:14.887988 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:14.887923 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:15.061481 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:15.061389 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:15.299215 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:15.299179 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:15.621862 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:15.621829 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:15.976296 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:15.976263 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:17.057578 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:17.057544 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:18.655445 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:18.655410 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:18.768659 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:18.768619 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:18.898459 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:18.898423 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:19.207741 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:19.207705 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:19.413582 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:19.413550 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:19.517738 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:19.517644 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:20.220354 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:20.220319 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:53:23.179153 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:23.179089 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:23.506663 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:23.506580 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:23.838740 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:23.838646 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:24.049427 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:24.049397 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:25.133153 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:25.133121 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:25.198734 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:25.198695 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:26.684675 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:26.684642 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:26.805120 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:26.805068 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:26.891552 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:26.891516 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:27.055605 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:27.055510 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:27.372990 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:27.372901 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:27.479868 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:27.479834 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:31.163462 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:31.163425 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:36.775265 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:36.775231 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:53:41.491485 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:41.491438 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:41.579206 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:41.579171 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:46.703261 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.703220 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g"] Jul 10 18:53:46.703642 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.703605 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8af1afdc-5110-4b63-bb48-aadb74bdf92b" containerName="main" Jul 10 18:53:46.703642 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.703616 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8af1afdc-5110-4b63-bb48-aadb74bdf92b" containerName="main" Jul 10 18:53:46.703642 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.703625 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e42f61d-30d8-4887-9bf3-d00526715a68" containerName="main" Jul 10 18:53:46.703642 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.703631 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e42f61d-30d8-4887-9bf3-d00526715a68" containerName="main" Jul 10 18:53:46.703776 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.703692 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8af1afdc-5110-4b63-bb48-aadb74bdf92b" containerName="main" Jul 10 18:53:46.703776 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.703704 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e42f61d-30d8-4887-9bf3-d00526715a68" containerName="main" Jul 10 18:53:46.706739 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.706719 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.709207 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.709183 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-afdf1e\"/\"openshift-service-ca.crt\"" Jul 10 18:53:46.710337 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.710307 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-afdf1e\"/\"default-dockercfg-vqscp\"" Jul 10 18:53:46.710337 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.710308 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-afdf1e\"/\"kube-root-ca.crt\"" Jul 10 18:53:46.710558 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.710309 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-afdf1e\"/\"auth-test-model-afdf1e-kserve-self-signed-certs\"" Jul 10 18:53:46.717900 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.717877 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g"] Jul 10 18:53:46.742638 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.742608 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:46.811542 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.811502 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc6c4\" (UniqueName: \"kubernetes.io/projected/4ef09cc3-08da-40d5-9097-f108b3b0164a-kube-api-access-gc6c4\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.811724 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.811600 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ef09cc3-08da-40d5-9097-f108b3b0164a-tls-certs\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.811724 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.811633 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-tmp-dir\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.811724 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.811706 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-home\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.811829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.811746 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-dshm\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.811829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.811786 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-model-cache\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.913034 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.912990 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-model-cache\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.913233 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.913079 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc6c4\" (UniqueName: \"kubernetes.io/projected/4ef09cc3-08da-40d5-9097-f108b3b0164a-kube-api-access-gc6c4\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.913233 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.913188 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ef09cc3-08da-40d5-9097-f108b3b0164a-tls-certs\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.913233 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.913218 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-tmp-dir\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.913361 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.913261 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-home\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.913361 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.913297 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-dshm\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.913456 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.913409 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-model-cache\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.913570 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.913554 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-tmp-dir\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.913620 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.913576 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-home\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.915507 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.915471 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-dshm\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.915684 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.915670 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ef09cc3-08da-40d5-9097-f108b3b0164a-tls-certs\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:46.922374 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:46.922346 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc6c4\" (UniqueName: \"kubernetes.io/projected/4ef09cc3-08da-40d5-9097-f108b3b0164a-kube-api-access-gc6c4\") pod \"auth-test-model-afdf1e-kserve-dc754679f-rnc4g\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:47.016542 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:47.016447 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:47.155899 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:47.155873 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g"] Jul 10 18:53:47.158145 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:53:47.158117 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ef09cc3_08da_40d5_9097_f108b3b0164a.slice/crio-e96372eab66bc0c1fcaadfa0c6f045a407e074b8c0ea48a01f6dfc3e5287c147 WatchSource:0}: Error finding container e96372eab66bc0c1fcaadfa0c6f045a407e074b8c0ea48a01f6dfc3e5287c147: Status 404 returned error can't find the container with id e96372eab66bc0c1fcaadfa0c6f045a407e074b8c0ea48a01f6dfc3e5287c147 Jul 10 18:53:47.877210 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:47.877164 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" event={"ID":"4ef09cc3-08da-40d5-9097-f108b3b0164a","Type":"ContainerStarted","Data":"a7724ac245132f69d818d91dd546fec04cc0360f783aa97697a9d3e3acebb2d5"} Jul 10 18:53:47.877664 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:47.877219 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" event={"ID":"4ef09cc3-08da-40d5-9097-f108b3b0164a","Type":"ContainerStarted","Data":"e96372eab66bc0c1fcaadfa0c6f045a407e074b8c0ea48a01f6dfc3e5287c147"} Jul 10 18:53:47.901997 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:47.901937 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" podStartSLOduration=1.901919238 podStartE2EDuration="1.901919238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:53:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:53:47.899012814 +0000 UTC m=+2187.312297596" watchObservedRunningTime="2026-07-10 18:53:47.901919238 +0000 UTC m=+2187.315204025" Jul 10 18:53:48.184845 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:48.184806 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:48.285859 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:48.285795 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:53:53.781022 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:53.780988 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:53:57.016864 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:57.016827 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:57.017278 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:57.016875 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:57.029891 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:57.029862 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:53:57.936679 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:53:57.936648 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:54:10.708160 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:10.708121 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:54:13.900285 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:13.900256 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg"] Jul 10 18:54:13.903588 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:13.903570 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:13.906609 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:13.906588 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f83faf\"/\"openshift-service-ca.crt\"" Jul 10 18:54:13.906765 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:13.906748 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f83faf\"/\"auth-test-model-f83faf-kserve-self-signed-certs\"" Jul 10 18:54:13.907740 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:13.907725 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f83faf\"/\"kube-root-ca.crt\"" Jul 10 18:54:13.907828 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:13.907790 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f83faf\"/\"default-dockercfg-8qpf9\"" Jul 10 18:54:13.922688 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:13.922658 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg"] Jul 10 18:54:13.966742 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:13.966711 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-dshm\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:13.966951 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:13.966812 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-home\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:13.966951 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:13.966867 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-model-cache\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:13.966951 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:13.966894 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-tmp-dir\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:13.967117 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:13.966953 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-tls-certs\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:13.967117 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:13.966996 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:54:13.967117 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:13.967044 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbww2\" (UniqueName: \"kubernetes.io/projected/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-kube-api-access-rbww2\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:14.068160 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:14.068121 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbww2\" (UniqueName: \"kubernetes.io/projected/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-kube-api-access-rbww2\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:14.068354 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:14.068196 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-dshm\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:14.068354 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:14.068243 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-home\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:14.068354 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:14.068339 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-model-cache\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:14.068516 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:14.068369 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-tmp-dir\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:14.068516 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:14.068404 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-tls-certs\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:14.068685 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:14.068665 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-home\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:14.068752 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:14.068684 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-model-cache\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:14.068819 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:14.068753 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-tmp-dir\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:14.070588 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:14.070569 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-dshm\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:14.070836 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:14.070816 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-tls-certs\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:14.077091 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:14.077069 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbww2\" (UniqueName: \"kubernetes.io/projected/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-kube-api-access-rbww2\") pod \"auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:14.213229 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:14.213184 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:14.355396 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:14.355368 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg"] Jul 10 18:54:14.357070 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:54:14.357046 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd9e4fbd_d115_4f92_bf7d_9f10f792752d.slice/crio-b2a4f5282795bac9a44d821ff1f00bf6cb45a0427d58da16e8f7a82cf2db7cd8 WatchSource:0}: Error finding container b2a4f5282795bac9a44d821ff1f00bf6cb45a0427d58da16e8f7a82cf2db7cd8: Status 404 returned error can't find the container with id b2a4f5282795bac9a44d821ff1f00bf6cb45a0427d58da16e8f7a82cf2db7cd8 Jul 10 18:54:15.019405 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:15.019373 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" event={"ID":"dd9e4fbd-d115-4f92-bf7d-9f10f792752d","Type":"ContainerStarted","Data":"39ebe689d8dcd62d6dd94c60f8405efcb0a06b119da2f6c3f079d81a50c53bfe"} Jul 10 18:54:15.019405 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:15.019410 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" event={"ID":"dd9e4fbd-d115-4f92-bf7d-9f10f792752d","Type":"ContainerStarted","Data":"b2a4f5282795bac9a44d821ff1f00bf6cb45a0427d58da16e8f7a82cf2db7cd8"} Jul 10 18:54:15.039773 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:15.039715 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" podStartSLOduration=2.039699916 podStartE2EDuration="2.039699916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:54:15.038004557 +0000 UTC m=+2214.451289344" watchObservedRunningTime="2026-07-10 18:54:15.039699916 +0000 UTC m=+2214.452984702" Jul 10 18:54:17.681701 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:17.681661 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:54:24.214081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:24.214034 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:24.214081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:24.214087 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:24.226780 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:24.226744 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:25.079925 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:25.079889 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:54:32.182086 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:32.182051 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:54:39.160319 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:39.160280 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:54:49.178887 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:54:49.178800 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:55:08.680106 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:55:08.680067 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:55:28.077697 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:55:28.077656 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:55:47.675257 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:55:47.675223 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:56:06.580895 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:06.580855 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:56:26.080714 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:26.080637 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:56:29.677206 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:29.677174 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g"] Jul 10 18:56:29.735535 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:29.735502 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:56:30.763830 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:30.763800 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg"] Jul 10 18:56:30.790796 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:30.790760 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:56:33.170142 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:33.170083 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:56:36.600776 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:36.600728 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:56:43.089233 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:43.089192 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:56:44.697739 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:44.697692 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" podUID="4ef09cc3-08da-40d5-9097-f108b3b0164a" containerName="main" containerID="cri-o://a7724ac245132f69d818d91dd546fec04cc0360f783aa97697a9d3e3acebb2d5" gracePeriod=45 Jul 10 18:56:44.952770 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:44.952703 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:56:45.029226 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.029188 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-tmp-dir\") pod \"4ef09cc3-08da-40d5-9097-f108b3b0164a\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " Jul 10 18:56:45.029431 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.029244 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-dshm\") pod \"4ef09cc3-08da-40d5-9097-f108b3b0164a\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " Jul 10 18:56:45.029431 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.029272 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-home\") pod \"4ef09cc3-08da-40d5-9097-f108b3b0164a\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " Jul 10 18:56:45.029431 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.029315 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ef09cc3-08da-40d5-9097-f108b3b0164a-tls-certs\") pod \"4ef09cc3-08da-40d5-9097-f108b3b0164a\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " Jul 10 18:56:45.029431 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.029357 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-model-cache\") pod \"4ef09cc3-08da-40d5-9097-f108b3b0164a\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " Jul 10 18:56:45.029431 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.029398 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gc6c4\" (UniqueName: \"kubernetes.io/projected/4ef09cc3-08da-40d5-9097-f108b3b0164a-kube-api-access-gc6c4\") pod \"4ef09cc3-08da-40d5-9097-f108b3b0164a\" (UID: \"4ef09cc3-08da-40d5-9097-f108b3b0164a\") " Jul 10 18:56:45.029707 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.029536 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4ef09cc3-08da-40d5-9097-f108b3b0164a" (UID: "4ef09cc3-08da-40d5-9097-f108b3b0164a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:56:45.029707 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.029625 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-home" (OuterVolumeSpecName: "home") pod "4ef09cc3-08da-40d5-9097-f108b3b0164a" (UID: "4ef09cc3-08da-40d5-9097-f108b3b0164a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:56:45.029707 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.029694 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-model-cache" (OuterVolumeSpecName: "model-cache") pod "4ef09cc3-08da-40d5-9097-f108b3b0164a" (UID: "4ef09cc3-08da-40d5-9097-f108b3b0164a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:56:45.029851 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.029758 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-home\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:56:45.029851 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.029777 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-tmp-dir\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:56:45.031678 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.031646 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-dshm" (OuterVolumeSpecName: "dshm") pod "4ef09cc3-08da-40d5-9097-f108b3b0164a" (UID: "4ef09cc3-08da-40d5-9097-f108b3b0164a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:56:45.031786 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.031696 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef09cc3-08da-40d5-9097-f108b3b0164a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4ef09cc3-08da-40d5-9097-f108b3b0164a" (UID: "4ef09cc3-08da-40d5-9097-f108b3b0164a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:56:45.031786 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.031710 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ef09cc3-08da-40d5-9097-f108b3b0164a-kube-api-access-gc6c4" (OuterVolumeSpecName: "kube-api-access-gc6c4") pod "4ef09cc3-08da-40d5-9097-f108b3b0164a" (UID: "4ef09cc3-08da-40d5-9097-f108b3b0164a"). InnerVolumeSpecName "kube-api-access-gc6c4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:56:45.130930 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.130894 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-dshm\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:56:45.130930 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.130924 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ef09cc3-08da-40d5-9097-f108b3b0164a-tls-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:56:45.130930 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.130935 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ef09cc3-08da-40d5-9097-f108b3b0164a-model-cache\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:56:45.131205 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.130944 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gc6c4\" (UniqueName: \"kubernetes.io/projected/4ef09cc3-08da-40d5-9097-f108b3b0164a-kube-api-access-gc6c4\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:56:45.763736 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.763700 2574 generic.go:356] "Generic (PLEG): container finished" podID="4ef09cc3-08da-40d5-9097-f108b3b0164a" containerID="a7724ac245132f69d818d91dd546fec04cc0360f783aa97697a9d3e3acebb2d5" exitCode=0 Jul 10 18:56:45.764269 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.763750 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" Jul 10 18:56:45.764269 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.763791 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" event={"ID":"4ef09cc3-08da-40d5-9097-f108b3b0164a","Type":"ContainerDied","Data":"a7724ac245132f69d818d91dd546fec04cc0360f783aa97697a9d3e3acebb2d5"} Jul 10 18:56:45.764269 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.763832 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g" event={"ID":"4ef09cc3-08da-40d5-9097-f108b3b0164a","Type":"ContainerDied","Data":"e96372eab66bc0c1fcaadfa0c6f045a407e074b8c0ea48a01f6dfc3e5287c147"} Jul 10 18:56:45.764269 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.763849 2574 scope.go:117] "RemoveContainer" containerID="a7724ac245132f69d818d91dd546fec04cc0360f783aa97697a9d3e3acebb2d5" Jul 10 18:56:45.772959 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.772936 2574 scope.go:117] "RemoveContainer" containerID="a7724ac245132f69d818d91dd546fec04cc0360f783aa97697a9d3e3acebb2d5" Jul 10 18:56:45.773314 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:56:45.773281 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7724ac245132f69d818d91dd546fec04cc0360f783aa97697a9d3e3acebb2d5\": container with ID starting with a7724ac245132f69d818d91dd546fec04cc0360f783aa97697a9d3e3acebb2d5 not found: ID does not exist" containerID="a7724ac245132f69d818d91dd546fec04cc0360f783aa97697a9d3e3acebb2d5" Jul 10 18:56:45.773423 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.773326 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7724ac245132f69d818d91dd546fec04cc0360f783aa97697a9d3e3acebb2d5"} err="failed to get container status \"a7724ac245132f69d818d91dd546fec04cc0360f783aa97697a9d3e3acebb2d5\": rpc error: code = NotFound desc = could not find container \"a7724ac245132f69d818d91dd546fec04cc0360f783aa97697a9d3e3acebb2d5\": container with ID starting with a7724ac245132f69d818d91dd546fec04cc0360f783aa97697a9d3e3acebb2d5 not found: ID does not exist" Jul 10 18:56:45.782317 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.782289 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g"] Jul 10 18:56:45.783956 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.783934 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" podUID="dd9e4fbd-d115-4f92-bf7d-9f10f792752d" containerName="main" containerID="cri-o://39ebe689d8dcd62d6dd94c60f8405efcb0a06b119da2f6c3f079d81a50c53bfe" gracePeriod=45 Jul 10 18:56:45.786840 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:45.786751 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/auth-test-model-afdf1e-kserve-dc754679f-rnc4g"] Jul 10 18:56:46.045379 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.045355 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:56:46.141344 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.141308 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-model-cache\") pod \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " Jul 10 18:56:46.141344 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.141346 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-dshm\") pod \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " Jul 10 18:56:46.141617 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.141402 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-tls-certs\") pod \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " Jul 10 18:56:46.141617 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.141457 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbww2\" (UniqueName: \"kubernetes.io/projected/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-kube-api-access-rbww2\") pod \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " Jul 10 18:56:46.141617 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.141486 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-tmp-dir\") pod \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " Jul 10 18:56:46.141617 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.141531 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-home\") pod \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\" (UID: \"dd9e4fbd-d115-4f92-bf7d-9f10f792752d\") " Jul 10 18:56:46.141822 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.141654 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-model-cache" (OuterVolumeSpecName: "model-cache") pod "dd9e4fbd-d115-4f92-bf7d-9f10f792752d" (UID: "dd9e4fbd-d115-4f92-bf7d-9f10f792752d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:56:46.141869 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.141818 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-home" (OuterVolumeSpecName: "home") pod "dd9e4fbd-d115-4f92-bf7d-9f10f792752d" (UID: "dd9e4fbd-d115-4f92-bf7d-9f10f792752d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:56:46.141869 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.141851 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dd9e4fbd-d115-4f92-bf7d-9f10f792752d" (UID: "dd9e4fbd-d115-4f92-bf7d-9f10f792752d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:56:46.141950 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.141937 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-home\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:56:46.141987 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.141953 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-model-cache\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:56:46.141987 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.141967 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-tmp-dir\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:56:46.143744 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.143716 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-dshm" (OuterVolumeSpecName: "dshm") pod "dd9e4fbd-d115-4f92-bf7d-9f10f792752d" (UID: "dd9e4fbd-d115-4f92-bf7d-9f10f792752d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:56:46.143744 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.143730 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-kube-api-access-rbww2" (OuterVolumeSpecName: "kube-api-access-rbww2") pod "dd9e4fbd-d115-4f92-bf7d-9f10f792752d" (UID: "dd9e4fbd-d115-4f92-bf7d-9f10f792752d"). InnerVolumeSpecName "kube-api-access-rbww2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:56:46.143905 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.143781 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dd9e4fbd-d115-4f92-bf7d-9f10f792752d" (UID: "dd9e4fbd-d115-4f92-bf7d-9f10f792752d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:56:46.242898 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.242856 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbww2\" (UniqueName: \"kubernetes.io/projected/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-kube-api-access-rbww2\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:56:46.242898 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.242889 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-dshm\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:56:46.242898 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.242905 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd9e4fbd-d115-4f92-bf7d-9f10f792752d-tls-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:56:46.770250 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.770219 2574 generic.go:356] "Generic (PLEG): container finished" podID="dd9e4fbd-d115-4f92-bf7d-9f10f792752d" containerID="39ebe689d8dcd62d6dd94c60f8405efcb0a06b119da2f6c3f079d81a50c53bfe" exitCode=0 Jul 10 18:56:46.770721 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.770262 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" Jul 10 18:56:46.770721 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.770306 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" event={"ID":"dd9e4fbd-d115-4f92-bf7d-9f10f792752d","Type":"ContainerDied","Data":"39ebe689d8dcd62d6dd94c60f8405efcb0a06b119da2f6c3f079d81a50c53bfe"} Jul 10 18:56:46.770721 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.770346 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg" event={"ID":"dd9e4fbd-d115-4f92-bf7d-9f10f792752d","Type":"ContainerDied","Data":"b2a4f5282795bac9a44d821ff1f00bf6cb45a0427d58da16e8f7a82cf2db7cd8"} Jul 10 18:56:46.770721 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.770362 2574 scope.go:117] "RemoveContainer" containerID="39ebe689d8dcd62d6dd94c60f8405efcb0a06b119da2f6c3f079d81a50c53bfe" Jul 10 18:56:46.779804 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.779784 2574 scope.go:117] "RemoveContainer" containerID="39ebe689d8dcd62d6dd94c60f8405efcb0a06b119da2f6c3f079d81a50c53bfe" Jul 10 18:56:46.780089 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:56:46.780067 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39ebe689d8dcd62d6dd94c60f8405efcb0a06b119da2f6c3f079d81a50c53bfe\": container with ID starting with 39ebe689d8dcd62d6dd94c60f8405efcb0a06b119da2f6c3f079d81a50c53bfe not found: ID does not exist" containerID="39ebe689d8dcd62d6dd94c60f8405efcb0a06b119da2f6c3f079d81a50c53bfe" Jul 10 18:56:46.780185 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.780116 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39ebe689d8dcd62d6dd94c60f8405efcb0a06b119da2f6c3f079d81a50c53bfe"} err="failed to get container status \"39ebe689d8dcd62d6dd94c60f8405efcb0a06b119da2f6c3f079d81a50c53bfe\": rpc error: code = NotFound desc = could not find container \"39ebe689d8dcd62d6dd94c60f8405efcb0a06b119da2f6c3f079d81a50c53bfe\": container with ID starting with 39ebe689d8dcd62d6dd94c60f8405efcb0a06b119da2f6c3f079d81a50c53bfe not found: ID does not exist" Jul 10 18:56:46.792883 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.792854 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg"] Jul 10 18:56:46.795705 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:46.795677 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/auth-test-model-f83faf-kserve-56ddb7db7f-m7jrg"] Jul 10 18:56:47.315699 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:47.315665 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ef09cc3-08da-40d5-9097-f108b3b0164a" path="/var/lib/kubelet/pods/4ef09cc3-08da-40d5-9097-f108b3b0164a/volumes" Jul 10 18:56:47.316055 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:47.316041 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd9e4fbd-d115-4f92-bf7d-9f10f792752d" path="/var/lib/kubelet/pods/dd9e4fbd-d115-4f92-bf7d-9f10f792752d/volumes" Jul 10 18:56:47.908226 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:47.908189 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:56:48.005761 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:48.005713 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:56:57.087286 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.087251 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm"] Jul 10 18:56:57.087642 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.087632 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd9e4fbd-d115-4f92-bf7d-9f10f792752d" containerName="main" Jul 10 18:56:57.087697 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.087643 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd9e4fbd-d115-4f92-bf7d-9f10f792752d" containerName="main" Jul 10 18:56:57.087697 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.087658 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ef09cc3-08da-40d5-9097-f108b3b0164a" containerName="main" Jul 10 18:56:57.087697 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.087663 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ef09cc3-08da-40d5-9097-f108b3b0164a" containerName="main" Jul 10 18:56:57.087798 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.087732 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd9e4fbd-d115-4f92-bf7d-9f10f792752d" containerName="main" Jul 10 18:56:57.087798 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.087742 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ef09cc3-08da-40d5-9097-f108b3b0164a" containerName="main" Jul 10 18:56:57.090733 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.090716 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.094231 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.094210 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f83faf\"/\"default-dockercfg-8qpf9\"" Jul 10 18:56:57.094985 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.094966 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f83faf\"/\"kube-root-ca.crt\"" Jul 10 18:56:57.095125 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.095058 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f83faf\"/\"openshift-service-ca.crt\"" Jul 10 18:56:57.095125 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.095088 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f83faf\"/\"sub-test-model-f83faf-kserve-self-signed-certs\"" Jul 10 18:56:57.104732 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.104704 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm"] Jul 10 18:56:57.127448 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.127412 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:56:57.246009 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.245966 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8nx7\" (UniqueName: \"kubernetes.io/projected/70e14827-0598-4cf5-a5fb-1b87f3c6af50-kube-api-access-n8nx7\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.246230 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.246078 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-home\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.246230 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.246127 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-model-cache\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.246230 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.246171 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-tmp-dir\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.246355 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.246246 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70e14827-0598-4cf5-a5fb-1b87f3c6af50-tls-certs\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.246355 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.246267 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-dshm\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.347746 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.347654 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70e14827-0598-4cf5-a5fb-1b87f3c6af50-tls-certs\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.347746 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.347694 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-dshm\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.347746 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.347730 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8nx7\" (UniqueName: \"kubernetes.io/projected/70e14827-0598-4cf5-a5fb-1b87f3c6af50-kube-api-access-n8nx7\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.348006 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.347789 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-home\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.348006 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.347813 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-model-cache\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.348006 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.347840 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-tmp-dir\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.348281 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.348256 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-home\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.348358 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.348263 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-tmp-dir\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.348358 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.348309 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-model-cache\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.349992 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.349972 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-dshm\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.350290 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.350273 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70e14827-0598-4cf5-a5fb-1b87f3c6af50-tls-certs\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.359542 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.359518 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8nx7\" (UniqueName: \"kubernetes.io/projected/70e14827-0598-4cf5-a5fb-1b87f3c6af50-kube-api-access-n8nx7\") pod \"sub-test-model-f83faf-kserve-6c8db7f567-htbxm\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.400616 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.400581 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:56:57.545104 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.545065 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm"] Jul 10 18:56:57.546826 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:56:57.546799 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70e14827_0598_4cf5_a5fb_1b87f3c6af50.slice/crio-fd57b5e89a4569f8506cf2c1c0e533a53066f25a2c46cb3d256df11ff84b6f29 WatchSource:0}: Error finding container fd57b5e89a4569f8506cf2c1c0e533a53066f25a2c46cb3d256df11ff84b6f29: Status 404 returned error can't find the container with id fd57b5e89a4569f8506cf2c1c0e533a53066f25a2c46cb3d256df11ff84b6f29 Jul 10 18:56:57.826477 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.826439 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" event={"ID":"70e14827-0598-4cf5-a5fb-1b87f3c6af50","Type":"ContainerStarted","Data":"580b7a76153d18425b6708ba01b20511f4027de480d736320f244872b7da9493"} Jul 10 18:56:57.826477 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.826490 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" event={"ID":"70e14827-0598-4cf5-a5fb-1b87f3c6af50","Type":"ContainerStarted","Data":"fd57b5e89a4569f8506cf2c1c0e533a53066f25a2c46cb3d256df11ff84b6f29"} Jul 10 18:56:57.847036 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:56:57.846980 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" podStartSLOduration=0.846964543 podStartE2EDuration="846.964543ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:56:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:56:57.84421688 +0000 UTC m=+2377.257501666" watchObservedRunningTime="2026-07-10 18:56:57.846964543 +0000 UTC m=+2377.260249329" Jul 10 18:57:03.183733 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:03.183696 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:57:07.401356 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:07.401316 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:57:07.401800 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:07.401369 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:57:07.413848 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:07.413817 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:57:07.883690 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:07.883661 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:57:21.281514 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:21.281482 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:57:21.284799 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:21.284776 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:57:21.285783 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:21.285762 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:57:21.289197 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:21.289170 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c9bx_0d031c2a-1702-4f4e-a90a-6e6767ebe663/ovn-acl-logging/0.log" Jul 10 18:57:21.767292 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:21.767260 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:57:21.868547 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:21.868515 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:57:25.671905 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:25.671873 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:57:31.544088 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:31.544054 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:57:32.486028 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:32.485989 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm"] Jul 10 18:57:32.505517 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:32.505487 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:57:39.476561 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.476527 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw"] Jul 10 18:57:39.479989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.479971 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.487183 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.486759 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-afdf1e\"/\"default-dockercfg-vqscp\"" Jul 10 18:57:39.487996 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.487969 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-afdf1e\"/\"rate-test-model-afdf1e-kserve-self-signed-certs\"" Jul 10 18:57:39.488136 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.487995 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-afdf1e\"/\"kube-root-ca.crt\"" Jul 10 18:57:39.488136 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.488073 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-afdf1e\"/\"openshift-service-ca.crt\"" Jul 10 18:57:39.492664 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.492566 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw"] Jul 10 18:57:39.588143 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.588076 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:57:39.648995 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.648956 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-tls-certs\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.649237 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.648999 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-home\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.649237 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.649147 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-tmp-dir\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.649237 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.649190 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-dshm\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.649237 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.649224 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-model-cache\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.649445 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.649295 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bvct\" (UniqueName: \"kubernetes.io/projected/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-kube-api-access-9bvct\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.750385 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.750290 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-tmp-dir\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.750385 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.750334 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-dshm\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.750385 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.750363 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-model-cache\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.750663 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.750403 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bvct\" (UniqueName: \"kubernetes.io/projected/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-kube-api-access-9bvct\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.750663 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.750462 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-tls-certs\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.750663 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.750496 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-home\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.750812 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.750784 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-tmp-dir\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.750859 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.750834 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-model-cache\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.750923 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.750907 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-home\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.752764 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.752745 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-dshm\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.752909 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.752890 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-tls-certs\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.758388 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.758364 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bvct\" (UniqueName: \"kubernetes.io/projected/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-kube-api-access-9bvct\") pod \"rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.795585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.795548 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:39.934223 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:39.934193 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw"] Jul 10 18:57:39.936531 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:57:39.936503 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf48768bb_db7c_4c0b_8869_dd11d8ba4d43.slice/crio-f2e4846e001c06d9a7159a735d59c95dc51d65750f452ea285532a04985e3f0f WatchSource:0}: Error finding container f2e4846e001c06d9a7159a735d59c95dc51d65750f452ea285532a04985e3f0f: Status 404 returned error can't find the container with id f2e4846e001c06d9a7159a735d59c95dc51d65750f452ea285532a04985e3f0f Jul 10 18:57:40.030516 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:40.030436 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" event={"ID":"f48768bb-db7c-4c0b-8869-dd11d8ba4d43","Type":"ContainerStarted","Data":"50c8080b3430dbbe269753f9b5a57dc077e3d33936f45f30c9ec89b77288c34b"} Jul 10 18:57:40.030516 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:40.030478 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" event={"ID":"f48768bb-db7c-4c0b-8869-dd11d8ba4d43","Type":"ContainerStarted","Data":"f2e4846e001c06d9a7159a735d59c95dc51d65750f452ea285532a04985e3f0f"} Jul 10 18:57:40.050526 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:40.050476 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" podStartSLOduration=1.05045957 podStartE2EDuration="1.05045957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:57:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:57:40.048243344 +0000 UTC m=+2419.461528162" watchObservedRunningTime="2026-07-10 18:57:40.05045957 +0000 UTC m=+2419.463744372" Jul 10 18:57:45.470592 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:45.470511 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:57:47.505559 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.505497 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" podUID="70e14827-0598-4cf5-a5fb-1b87f3c6af50" containerName="main" containerID="cri-o://580b7a76153d18425b6708ba01b20511f4027de480d736320f244872b7da9493" gracePeriod=45 Jul 10 18:57:47.770874 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.770806 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:57:47.929167 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.929132 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70e14827-0598-4cf5-a5fb-1b87f3c6af50-tls-certs\") pod \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " Jul 10 18:57:47.929351 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.929211 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-tmp-dir\") pod \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " Jul 10 18:57:47.929351 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.929244 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-model-cache\") pod \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " Jul 10 18:57:47.929351 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.929275 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8nx7\" (UniqueName: \"kubernetes.io/projected/70e14827-0598-4cf5-a5fb-1b87f3c6af50-kube-api-access-n8nx7\") pod \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " Jul 10 18:57:47.929351 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.929297 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-dshm\") pod \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " Jul 10 18:57:47.929571 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.929382 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-home\") pod \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\" (UID: \"70e14827-0598-4cf5-a5fb-1b87f3c6af50\") " Jul 10 18:57:47.929571 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.929516 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "70e14827-0598-4cf5-a5fb-1b87f3c6af50" (UID: "70e14827-0598-4cf5-a5fb-1b87f3c6af50"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:57:47.929571 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.929531 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-model-cache" (OuterVolumeSpecName: "model-cache") pod "70e14827-0598-4cf5-a5fb-1b87f3c6af50" (UID: "70e14827-0598-4cf5-a5fb-1b87f3c6af50"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:57:47.929699 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.929609 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-home" (OuterVolumeSpecName: "home") pod "70e14827-0598-4cf5-a5fb-1b87f3c6af50" (UID: "70e14827-0598-4cf5-a5fb-1b87f3c6af50"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:57:47.929772 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.929752 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-tmp-dir\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:57:47.929810 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.929778 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-model-cache\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:57:47.929810 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.929789 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-home\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:57:47.931385 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.931361 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-dshm" (OuterVolumeSpecName: "dshm") pod "70e14827-0598-4cf5-a5fb-1b87f3c6af50" (UID: "70e14827-0598-4cf5-a5fb-1b87f3c6af50"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:57:47.931488 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.931406 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70e14827-0598-4cf5-a5fb-1b87f3c6af50-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "70e14827-0598-4cf5-a5fb-1b87f3c6af50" (UID: "70e14827-0598-4cf5-a5fb-1b87f3c6af50"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:57:47.931488 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:47.931447 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70e14827-0598-4cf5-a5fb-1b87f3c6af50-kube-api-access-n8nx7" (OuterVolumeSpecName: "kube-api-access-n8nx7") pod "70e14827-0598-4cf5-a5fb-1b87f3c6af50" (UID: "70e14827-0598-4cf5-a5fb-1b87f3c6af50"). InnerVolumeSpecName "kube-api-access-n8nx7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:57:48.030743 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:48.030653 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8nx7\" (UniqueName: \"kubernetes.io/projected/70e14827-0598-4cf5-a5fb-1b87f3c6af50-kube-api-access-n8nx7\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:57:48.030743 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:48.030685 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70e14827-0598-4cf5-a5fb-1b87f3c6af50-dshm\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:57:48.030743 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:48.030695 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70e14827-0598-4cf5-a5fb-1b87f3c6af50-tls-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:57:48.073164 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:48.073128 2574 generic.go:356] "Generic (PLEG): container finished" podID="70e14827-0598-4cf5-a5fb-1b87f3c6af50" containerID="580b7a76153d18425b6708ba01b20511f4027de480d736320f244872b7da9493" exitCode=0 Jul 10 18:57:48.073164 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:48.073169 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" Jul 10 18:57:48.073410 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:48.073216 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" event={"ID":"70e14827-0598-4cf5-a5fb-1b87f3c6af50","Type":"ContainerDied","Data":"580b7a76153d18425b6708ba01b20511f4027de480d736320f244872b7da9493"} Jul 10 18:57:48.073410 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:48.073260 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm" event={"ID":"70e14827-0598-4cf5-a5fb-1b87f3c6af50","Type":"ContainerDied","Data":"fd57b5e89a4569f8506cf2c1c0e533a53066f25a2c46cb3d256df11ff84b6f29"} Jul 10 18:57:48.073410 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:48.073278 2574 scope.go:117] "RemoveContainer" containerID="580b7a76153d18425b6708ba01b20511f4027de480d736320f244872b7da9493" Jul 10 18:57:48.082805 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:48.082781 2574 scope.go:117] "RemoveContainer" containerID="580b7a76153d18425b6708ba01b20511f4027de480d736320f244872b7da9493" Jul 10 18:57:48.083113 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:57:48.083070 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"580b7a76153d18425b6708ba01b20511f4027de480d736320f244872b7da9493\": container with ID starting with 580b7a76153d18425b6708ba01b20511f4027de480d736320f244872b7da9493 not found: ID does not exist" containerID="580b7a76153d18425b6708ba01b20511f4027de480d736320f244872b7da9493" Jul 10 18:57:48.083183 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:48.083122 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"580b7a76153d18425b6708ba01b20511f4027de480d736320f244872b7da9493"} err="failed to get container status \"580b7a76153d18425b6708ba01b20511f4027de480d736320f244872b7da9493\": rpc error: code = NotFound desc = could not find container \"580b7a76153d18425b6708ba01b20511f4027de480d736320f244872b7da9493\": container with ID starting with 580b7a76153d18425b6708ba01b20511f4027de480d736320f244872b7da9493 not found: ID does not exist" Jul 10 18:57:48.095471 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:48.095440 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm"] Jul 10 18:57:48.099207 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:48.099177 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/sub-test-model-f83faf-kserve-6c8db7f567-htbxm"] Jul 10 18:57:49.315007 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:49.314972 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70e14827-0598-4cf5-a5fb-1b87f3c6af50" path="/var/lib/kubelet/pods/70e14827-0598-4cf5-a5fb-1b87f3c6af50/volumes" Jul 10 18:57:49.796461 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:49.796426 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:49.796461 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:49.796475 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:49.808867 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:49.808839 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:57:50.095070 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:57:50.094980 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:58:01.893725 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.893687 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl"] Jul 10 18:58:01.894148 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.894122 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70e14827-0598-4cf5-a5fb-1b87f3c6af50" containerName="main" Jul 10 18:58:01.894148 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.894139 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="70e14827-0598-4cf5-a5fb-1b87f3c6af50" containerName="main" Jul 10 18:58:01.894236 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.894213 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="70e14827-0598-4cf5-a5fb-1b87f3c6af50" containerName="main" Jul 10 18:58:01.897335 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.897316 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:01.901055 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.901029 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f83faf\"/\"openshift-service-ca.crt\"" Jul 10 18:58:01.901812 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.901788 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f83faf\"/\"rate-test-model-f83faf-kserve-self-signed-certs\"" Jul 10 18:58:01.902276 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.902246 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f83faf\"/\"default-dockercfg-8qpf9\"" Jul 10 18:58:01.902375 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.902326 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f83faf\"/\"kube-root-ca.crt\"" Jul 10 18:58:01.909003 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.908980 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl"] Jul 10 18:58:01.935317 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.935285 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:58:01.973444 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.973403 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1930ec1c-3f14-4764-8581-f3e09fe76ad7-tls-certs\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:01.973636 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.973469 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-tmp-dir\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:01.973636 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.973487 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-model-cache\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:01.973636 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.973526 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-dshm\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:01.973636 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.973545 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7s8f\" (UniqueName: \"kubernetes.io/projected/1930ec1c-3f14-4764-8581-f3e09fe76ad7-kube-api-access-t7s8f\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:01.973636 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:01.973570 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-home\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:02.074331 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:02.074296 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-dshm\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:02.074331 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:02.074337 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7s8f\" (UniqueName: \"kubernetes.io/projected/1930ec1c-3f14-4764-8581-f3e09fe76ad7-kube-api-access-t7s8f\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:02.074570 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:02.074469 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-home\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:02.074570 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:02.074550 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1930ec1c-3f14-4764-8581-f3e09fe76ad7-tls-certs\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:02.074676 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:02.074611 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-tmp-dir\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:02.074676 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:02.074633 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-model-cache\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:02.074896 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:02.074876 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-home\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:02.074978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:02.074954 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-tmp-dir\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:02.075046 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:02.074997 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-model-cache\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:02.076831 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:02.076801 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-dshm\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:02.076970 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:02.076952 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1930ec1c-3f14-4764-8581-f3e09fe76ad7-tls-certs\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:02.084208 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:02.084180 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7s8f\" (UniqueName: \"kubernetes.io/projected/1930ec1c-3f14-4764-8581-f3e09fe76ad7-kube-api-access-t7s8f\") pod \"rate-test-model-f83faf-kserve-8985fc4c9-jw7gl\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:02.208755 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:02.208703 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:02.354465 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:02.354433 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl"] Jul 10 18:58:02.356488 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:58:02.356457 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1930ec1c_3f14_4764_8581_f3e09fe76ad7.slice/crio-86a8fbdfd5b59d3b04efccb4e0896101d20b3024884daf8a786a836ff02dbf90 WatchSource:0}: Error finding container 86a8fbdfd5b59d3b04efccb4e0896101d20b3024884daf8a786a836ff02dbf90: Status 404 returned error can't find the container with id 86a8fbdfd5b59d3b04efccb4e0896101d20b3024884daf8a786a836ff02dbf90 Jul 10 18:58:03.145160 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:03.145121 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" event={"ID":"1930ec1c-3f14-4764-8581-f3e09fe76ad7","Type":"ContainerStarted","Data":"027491ac816b5ba7b3b00db61517a9a4933f47110ec2f453ec0b57725fec2e1e"} Jul 10 18:58:03.145647 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:03.145173 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" event={"ID":"1930ec1c-3f14-4764-8581-f3e09fe76ad7","Type":"ContainerStarted","Data":"86a8fbdfd5b59d3b04efccb4e0896101d20b3024884daf8a786a836ff02dbf90"} Jul 10 18:58:03.168725 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:03.168665 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" podStartSLOduration=2.168649011 podStartE2EDuration="2.168649011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:58:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:58:03.167509278 +0000 UTC m=+2442.580794062" watchObservedRunningTime="2026-07-10 18:58:03.168649011 +0000 UTC m=+2442.581933796" Jul 10 18:58:06.283517 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:06.283479 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:58:12.209180 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:12.209129 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:12.209627 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:12.209194 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:12.221483 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:12.221458 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:13.209022 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:13.208992 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:58:21.690075 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:21.690033 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:58:42.181771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:42.181742 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:58:42.465305 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:42.465230 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:58:42.564726 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:42.564692 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:58:49.024209 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:49.024175 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw"] Jul 10 18:58:49.024866 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:49.024844 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:58:49.118234 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:49.118204 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:58:50.092141 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:50.092108 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl"] Jul 10 18:58:50.109138 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:50.109087 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:58:51.090131 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:51.090082 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7669-139946651437872"] Jul 10 18:58:51.093654 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:51.093632 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946651437872" Jul 10 18:58:51.096420 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:51.096403 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-fc7jf\"" Jul 10 18:58:51.106467 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:51.106441 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7669-139946651437872"] Jul 10 18:58:51.263661 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:51.263619 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jmf6\" (UniqueName: \"kubernetes.io/projected/3f136337-b08a-41fa-9134-a3039c2e9246-kube-api-access-5jmf6\") pod \"test-curl-7669-139946651437872\" (UID: \"3f136337-b08a-41fa-9134-a3039c2e9246\") " pod="openshift-ingress/test-curl-7669-139946651437872" Jul 10 18:58:51.365136 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:51.365003 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jmf6\" (UniqueName: \"kubernetes.io/projected/3f136337-b08a-41fa-9134-a3039c2e9246-kube-api-access-5jmf6\") pod \"test-curl-7669-139946651437872\" (UID: \"3f136337-b08a-41fa-9134-a3039c2e9246\") " pod="openshift-ingress/test-curl-7669-139946651437872" Jul 10 18:58:51.374140 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:51.374114 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jmf6\" (UniqueName: \"kubernetes.io/projected/3f136337-b08a-41fa-9134-a3039c2e9246-kube-api-access-5jmf6\") pod \"test-curl-7669-139946651437872\" (UID: \"3f136337-b08a-41fa-9134-a3039c2e9246\") " pod="openshift-ingress/test-curl-7669-139946651437872" Jul 10 18:58:51.402676 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:51.402629 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946651437872" Jul 10 18:58:51.538739 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:51.538696 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7669-139946651437872"] Jul 10 18:58:51.540898 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:58:51.540864 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f136337_b08a_41fa_9134_a3039c2e9246.slice/crio-cac6c3fa9cb2d312149ea07ba0a2b19cb17f6d654b8a5e7901868a29c113baf6 WatchSource:0}: Error finding container cac6c3fa9cb2d312149ea07ba0a2b19cb17f6d654b8a5e7901868a29c113baf6: Status 404 returned error can't find the container with id cac6c3fa9cb2d312149ea07ba0a2b19cb17f6d654b8a5e7901868a29c113baf6 Jul 10 18:58:51.542353 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:51.542334 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:58:52.395853 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:52.395817 2574 generic.go:356] "Generic (PLEG): container finished" podID="3f136337-b08a-41fa-9134-a3039c2e9246" containerID="ae1122c9ab7bf7de008b4f50f6004bfd65ca21fd26f67304c4f4e8ed25bd0513" exitCode=0 Jul 10 18:58:52.396454 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:52.395864 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7669-139946651437872" event={"ID":"3f136337-b08a-41fa-9134-a3039c2e9246","Type":"ContainerDied","Data":"ae1122c9ab7bf7de008b4f50f6004bfd65ca21fd26f67304c4f4e8ed25bd0513"} Jul 10 18:58:52.396454 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:52.395891 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7669-139946651437872" event={"ID":"3f136337-b08a-41fa-9134-a3039c2e9246","Type":"ContainerStarted","Data":"cac6c3fa9cb2d312149ea07ba0a2b19cb17f6d654b8a5e7901868a29c113baf6"} Jul 10 18:58:53.547347 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.547314 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946651437872" Jul 10 18:58:53.570995 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.570964 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7669-139946651437872_3f136337-b08a-41fa-9134-a3039c2e9246/test-curl-7669-139946651437872/0.log" Jul 10 18:58:53.598498 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.598467 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7669-139946651437872"] Jul 10 18:58:53.603754 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.603728 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7669-139946651437872"] Jul 10 18:58:53.685213 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.685182 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jmf6\" (UniqueName: \"kubernetes.io/projected/3f136337-b08a-41fa-9134-a3039c2e9246-kube-api-access-5jmf6\") pod \"3f136337-b08a-41fa-9134-a3039c2e9246\" (UID: \"3f136337-b08a-41fa-9134-a3039c2e9246\") " Jul 10 18:58:53.687531 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.687492 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f136337-b08a-41fa-9134-a3039c2e9246-kube-api-access-5jmf6" (OuterVolumeSpecName: "kube-api-access-5jmf6") pod "3f136337-b08a-41fa-9134-a3039c2e9246" (UID: "3f136337-b08a-41fa-9134-a3039c2e9246"). InnerVolumeSpecName "kube-api-access-5jmf6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:58:53.727448 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.727409 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7669-139946651438768"] Jul 10 18:58:53.727799 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.727787 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f136337-b08a-41fa-9134-a3039c2e9246" containerName="test-curl-7669-139946651437872" Jul 10 18:58:53.727842 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.727801 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f136337-b08a-41fa-9134-a3039c2e9246" containerName="test-curl-7669-139946651437872" Jul 10 18:58:53.727891 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.727882 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f136337-b08a-41fa-9134-a3039c2e9246" containerName="test-curl-7669-139946651437872" Jul 10 18:58:53.730982 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.730964 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946651438768" Jul 10 18:58:53.740299 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.740274 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7669-139946651438768"] Jul 10 18:58:53.786718 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.786681 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jmf6\" (UniqueName: \"kubernetes.io/projected/3f136337-b08a-41fa-9134-a3039c2e9246-kube-api-access-5jmf6\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:58:53.888028 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.887984 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wltsh\" (UniqueName: \"kubernetes.io/projected/db92bd24-b89b-43f4-9e7c-3968d72e1ba5-kube-api-access-wltsh\") pod \"test-curl-7669-139946651438768\" (UID: \"db92bd24-b89b-43f4-9e7c-3968d72e1ba5\") " pod="openshift-ingress/test-curl-7669-139946651438768" Jul 10 18:58:53.989076 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.988977 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wltsh\" (UniqueName: \"kubernetes.io/projected/db92bd24-b89b-43f4-9e7c-3968d72e1ba5-kube-api-access-wltsh\") pod \"test-curl-7669-139946651438768\" (UID: \"db92bd24-b89b-43f4-9e7c-3968d72e1ba5\") " pod="openshift-ingress/test-curl-7669-139946651438768" Jul 10 18:58:53.998503 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:53.998472 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wltsh\" (UniqueName: \"kubernetes.io/projected/db92bd24-b89b-43f4-9e7c-3968d72e1ba5-kube-api-access-wltsh\") pod \"test-curl-7669-139946651438768\" (UID: \"db92bd24-b89b-43f4-9e7c-3968d72e1ba5\") " pod="openshift-ingress/test-curl-7669-139946651438768" Jul 10 18:58:54.041213 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:54.041182 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946651438768" Jul 10 18:58:54.199972 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:54.199936 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7669-139946651438768"] Jul 10 18:58:54.202823 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:58:54.202796 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb92bd24_b89b_43f4_9e7c_3968d72e1ba5.slice/crio-8b59142e8108920be62818809bbe01445f7c8fcfdd4990264e695ca3710ec7dd WatchSource:0}: Error finding container 8b59142e8108920be62818809bbe01445f7c8fcfdd4990264e695ca3710ec7dd: Status 404 returned error can't find the container with id 8b59142e8108920be62818809bbe01445f7c8fcfdd4990264e695ca3710ec7dd Jul 10 18:58:54.406351 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:54.406255 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946651437872" Jul 10 18:58:54.406351 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:54.406293 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cac6c3fa9cb2d312149ea07ba0a2b19cb17f6d654b8a5e7901868a29c113baf6" Jul 10 18:58:54.407562 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:54.407536 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7669-139946651438768" event={"ID":"db92bd24-b89b-43f4-9e7c-3968d72e1ba5","Type":"ContainerStarted","Data":"8b59142e8108920be62818809bbe01445f7c8fcfdd4990264e695ca3710ec7dd"} Jul 10 18:58:54.417702 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:58:54.417670 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7669-139946651437872\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="3f136337-b08a-41fa-9134-a3039c2e9246" pod="openshift-ingress/test-curl-7669-139946651437872" Jul 10 18:58:55.316075 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:55.316038 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f136337-b08a-41fa-9134-a3039c2e9246" path="/var/lib/kubelet/pods/3f136337-b08a-41fa-9134-a3039c2e9246/volumes" Jul 10 18:58:55.414361 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:55.414324 2574 generic.go:356] "Generic (PLEG): container finished" podID="db92bd24-b89b-43f4-9e7c-3968d72e1ba5" containerID="f963f45339a94d3027daa694be4bee3778e8967b5cbcf18196fb4424fddc192b" exitCode=0 Jul 10 18:58:55.414539 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:55.414403 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7669-139946651438768" event={"ID":"db92bd24-b89b-43f4-9e7c-3968d72e1ba5","Type":"ContainerDied","Data":"f963f45339a94d3027daa694be4bee3778e8967b5cbcf18196fb4424fddc192b"} Jul 10 18:58:56.566870 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:56.566842 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946651438768" Jul 10 18:58:56.589256 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:56.589224 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7669-139946651438768_db92bd24-b89b-43f4-9e7c-3968d72e1ba5/test-curl-7669-139946651438768/0.log" Jul 10 18:58:56.614920 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:56.614873 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7669-139946651438768"] Jul 10 18:58:56.616739 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:56.616714 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7669-139946651438768"] Jul 10 18:58:56.715836 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:56.715798 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wltsh\" (UniqueName: \"kubernetes.io/projected/db92bd24-b89b-43f4-9e7c-3968d72e1ba5-kube-api-access-wltsh\") pod \"db92bd24-b89b-43f4-9e7c-3968d72e1ba5\" (UID: \"db92bd24-b89b-43f4-9e7c-3968d72e1ba5\") " Jul 10 18:58:56.717954 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:56.717917 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db92bd24-b89b-43f4-9e7c-3968d72e1ba5-kube-api-access-wltsh" (OuterVolumeSpecName: "kube-api-access-wltsh") pod "db92bd24-b89b-43f4-9e7c-3968d72e1ba5" (UID: "db92bd24-b89b-43f4-9e7c-3968d72e1ba5"). InnerVolumeSpecName "kube-api-access-wltsh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:58:56.816905 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:56.816865 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wltsh\" (UniqueName: \"kubernetes.io/projected/db92bd24-b89b-43f4-9e7c-3968d72e1ba5-kube-api-access-wltsh\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:58:57.315873 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:57.315842 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db92bd24-b89b-43f4-9e7c-3968d72e1ba5" path="/var/lib/kubelet/pods/db92bd24-b89b-43f4-9e7c-3968d72e1ba5/volumes" Jul 10 18:58:57.425690 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:57.425664 2574 scope.go:117] "RemoveContainer" containerID="f963f45339a94d3027daa694be4bee3778e8967b5cbcf18196fb4424fddc192b" Jul 10 18:58:57.425690 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:57.425679 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946651438768" Jul 10 18:58:57.874678 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:57.874641 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:58:59.948143 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:59.948106 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7669-139946653554560"] Jul 10 18:58:59.948535 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:59.948485 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db92bd24-b89b-43f4-9e7c-3968d72e1ba5" containerName="test-curl-7669-139946651438768" Jul 10 18:58:59.948535 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:59.948496 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db92bd24-b89b-43f4-9e7c-3968d72e1ba5" containerName="test-curl-7669-139946651438768" Jul 10 18:58:59.948612 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:59.948572 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="db92bd24-b89b-43f4-9e7c-3968d72e1ba5" containerName="test-curl-7669-139946651438768" Jul 10 18:58:59.952583 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:59.952565 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:58:59.955477 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:59.955456 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-fc7jf\"" Jul 10 18:58:59.962697 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:58:59.962674 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7669-139946653554560"] Jul 10 18:59:00.047585 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:00.047548 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckxxp\" (UniqueName: \"kubernetes.io/projected/38b04f48-404a-4850-b0cd-79a61574ad3c-kube-api-access-ckxxp\") pod \"test-curl-7669-139946653554560\" (UID: \"38b04f48-404a-4850-b0cd-79a61574ad3c\") " pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:59:00.148580 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:00.148542 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckxxp\" (UniqueName: \"kubernetes.io/projected/38b04f48-404a-4850-b0cd-79a61574ad3c-kube-api-access-ckxxp\") pod \"test-curl-7669-139946653554560\" (UID: \"38b04f48-404a-4850-b0cd-79a61574ad3c\") " pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:59:00.157645 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:00.157614 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckxxp\" (UniqueName: \"kubernetes.io/projected/38b04f48-404a-4850-b0cd-79a61574ad3c-kube-api-access-ckxxp\") pod \"test-curl-7669-139946653554560\" (UID: \"38b04f48-404a-4850-b0cd-79a61574ad3c\") " pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:59:00.261637 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:00.261545 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:59:00.404178 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:00.404150 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7669-139946653554560"] Jul 10 18:59:00.406320 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:59:00.406292 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38b04f48_404a_4850_b0cd_79a61574ad3c.slice/crio-fd4f83abc0cb775e660621d19bafe6f8fad066bf71c9b9a97452496dbcfdcc13 WatchSource:0}: Error finding container fd4f83abc0cb775e660621d19bafe6f8fad066bf71c9b9a97452496dbcfdcc13: Status 404 returned error can't find the container with id fd4f83abc0cb775e660621d19bafe6f8fad066bf71c9b9a97452496dbcfdcc13 Jul 10 18:59:00.443256 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:00.443224 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7669-139946653554560" event={"ID":"38b04f48-404a-4850-b0cd-79a61574ad3c","Type":"ContainerStarted","Data":"fd4f83abc0cb775e660621d19bafe6f8fad066bf71c9b9a97452496dbcfdcc13"} Jul 10 18:59:01.449963 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:01.449928 2574 generic.go:356] "Generic (PLEG): container finished" podID="38b04f48-404a-4850-b0cd-79a61574ad3c" containerID="74aeefd618cc4181c7970ea73db99325567ff60c1470a30e2841b7401d99582d" exitCode=0 Jul 10 18:59:01.450402 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:01.449977 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7669-139946653554560" event={"ID":"38b04f48-404a-4850-b0cd-79a61574ad3c","Type":"ContainerDied","Data":"74aeefd618cc4181c7970ea73db99325567ff60c1470a30e2841b7401d99582d"} Jul 10 18:59:02.595574 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.595545 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:59:02.626081 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.626057 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7669-139946653554560_38b04f48-404a-4850-b0cd-79a61574ad3c/test-curl-7669-139946653554560/0.log" Jul 10 18:59:02.658269 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.658228 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7669-139946653554560"] Jul 10 18:59:02.663833 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.663803 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7669-139946653554560"] Jul 10 18:59:02.671012 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.670985 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckxxp\" (UniqueName: \"kubernetes.io/projected/38b04f48-404a-4850-b0cd-79a61574ad3c-kube-api-access-ckxxp\") pod \"38b04f48-404a-4850-b0cd-79a61574ad3c\" (UID: \"38b04f48-404a-4850-b0cd-79a61574ad3c\") " Jul 10 18:59:02.673132 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.673105 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38b04f48-404a-4850-b0cd-79a61574ad3c-kube-api-access-ckxxp" (OuterVolumeSpecName: "kube-api-access-ckxxp") pod "38b04f48-404a-4850-b0cd-79a61574ad3c" (UID: "38b04f48-404a-4850-b0cd-79a61574ad3c"). InnerVolumeSpecName "kube-api-access-ckxxp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:59:02.771766 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.771668 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckxxp\" (UniqueName: \"kubernetes.io/projected/38b04f48-404a-4850-b0cd-79a61574ad3c-kube-api-access-ckxxp\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:02.788603 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.788566 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7669-139946653555568"] Jul 10 18:59:02.788978 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.788963 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38b04f48-404a-4850-b0cd-79a61574ad3c" containerName="test-curl-7669-139946653554560" Jul 10 18:59:02.789024 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.788979 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="38b04f48-404a-4850-b0cd-79a61574ad3c" containerName="test-curl-7669-139946653554560" Jul 10 18:59:02.789072 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.789049 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="38b04f48-404a-4850-b0cd-79a61574ad3c" containerName="test-curl-7669-139946653554560" Jul 10 18:59:02.792011 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.791994 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653555568" Jul 10 18:59:02.797353 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.797328 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7669-139946653555568"] Jul 10 18:59:02.872278 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.872243 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-447h9\" (UniqueName: \"kubernetes.io/projected/edeac1b8-36da-41de-9dbf-5e4424e5bac0-kube-api-access-447h9\") pod \"test-curl-7669-139946653555568\" (UID: \"edeac1b8-36da-41de-9dbf-5e4424e5bac0\") " pod="openshift-ingress/test-curl-7669-139946653555568" Jul 10 18:59:02.973025 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.972993 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-447h9\" (UniqueName: \"kubernetes.io/projected/edeac1b8-36da-41de-9dbf-5e4424e5bac0-kube-api-access-447h9\") pod \"test-curl-7669-139946653555568\" (UID: \"edeac1b8-36da-41de-9dbf-5e4424e5bac0\") " pod="openshift-ingress/test-curl-7669-139946653555568" Jul 10 18:59:02.982447 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:02.982417 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-447h9\" (UniqueName: \"kubernetes.io/projected/edeac1b8-36da-41de-9dbf-5e4424e5bac0-kube-api-access-447h9\") pod \"test-curl-7669-139946653555568\" (UID: \"edeac1b8-36da-41de-9dbf-5e4424e5bac0\") " pod="openshift-ingress/test-curl-7669-139946653555568" Jul 10 18:59:03.102658 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:03.102569 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653555568" Jul 10 18:59:03.244295 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:03.244258 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7669-139946653555568"] Jul 10 18:59:03.247685 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:59:03.247651 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podedeac1b8_36da_41de_9dbf_5e4424e5bac0.slice/crio-ab56babc0460b5733b78592acbc604c72bc08105677bff0632cde0de0aec9c8e WatchSource:0}: Error finding container ab56babc0460b5733b78592acbc604c72bc08105677bff0632cde0de0aec9c8e: Status 404 returned error can't find the container with id ab56babc0460b5733b78592acbc604c72bc08105677bff0632cde0de0aec9c8e Jul 10 18:59:03.317160 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:03.317126 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38b04f48-404a-4850-b0cd-79a61574ad3c" path="/var/lib/kubelet/pods/38b04f48-404a-4850-b0cd-79a61574ad3c/volumes" Jul 10 18:59:03.461902 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:03.461871 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:59:03.462116 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:03.461921 2574 scope.go:117] "RemoveContainer" containerID="74aeefd618cc4181c7970ea73db99325567ff60c1470a30e2841b7401d99582d" Jul 10 18:59:03.463546 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:03.463525 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7669-139946653555568" event={"ID":"edeac1b8-36da-41de-9dbf-5e4424e5bac0","Type":"ContainerStarted","Data":"ab56babc0460b5733b78592acbc604c72bc08105677bff0632cde0de0aec9c8e"} Jul 10 18:59:04.044925 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.044891 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" podUID="f48768bb-db7c-4c0b-8869-dd11d8ba4d43" containerName="main" containerID="cri-o://50c8080b3430dbbe269753f9b5a57dc077e3d33936f45f30c9ec89b77288c34b" gracePeriod=45 Jul 10 18:59:04.296731 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.296673 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:59:04.385566 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.385535 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-home\") pod \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " Jul 10 18:59:04.385566 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.385574 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bvct\" (UniqueName: \"kubernetes.io/projected/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-kube-api-access-9bvct\") pod \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " Jul 10 18:59:04.385827 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.385604 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-dshm\") pod \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " Jul 10 18:59:04.385827 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.385644 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-model-cache\") pod \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " Jul 10 18:59:04.385827 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.385700 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-tmp-dir\") pod \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " Jul 10 18:59:04.385827 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.385731 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-tls-certs\") pod \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\" (UID: \"f48768bb-db7c-4c0b-8869-dd11d8ba4d43\") " Jul 10 18:59:04.386017 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.385874 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-home" (OuterVolumeSpecName: "home") pod "f48768bb-db7c-4c0b-8869-dd11d8ba4d43" (UID: "f48768bb-db7c-4c0b-8869-dd11d8ba4d43"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:04.386017 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.385895 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-model-cache" (OuterVolumeSpecName: "model-cache") pod "f48768bb-db7c-4c0b-8869-dd11d8ba4d43" (UID: "f48768bb-db7c-4c0b-8869-dd11d8ba4d43"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:04.387739 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.387703 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f48768bb-db7c-4c0b-8869-dd11d8ba4d43" (UID: "f48768bb-db7c-4c0b-8869-dd11d8ba4d43"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:04.388250 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.388225 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-dshm" (OuterVolumeSpecName: "dshm") pod "f48768bb-db7c-4c0b-8869-dd11d8ba4d43" (UID: "f48768bb-db7c-4c0b-8869-dd11d8ba4d43"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:04.388893 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.388718 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-kube-api-access-9bvct" (OuterVolumeSpecName: "kube-api-access-9bvct") pod "f48768bb-db7c-4c0b-8869-dd11d8ba4d43" (UID: "f48768bb-db7c-4c0b-8869-dd11d8ba4d43"). InnerVolumeSpecName "kube-api-access-9bvct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:59:04.388989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.388948 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-home\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:04.388989 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.388968 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-model-cache\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:04.395510 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.395486 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f48768bb-db7c-4c0b-8869-dd11d8ba4d43" (UID: "f48768bb-db7c-4c0b-8869-dd11d8ba4d43"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:59:04.469267 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.469236 2574 generic.go:356] "Generic (PLEG): container finished" podID="f48768bb-db7c-4c0b-8869-dd11d8ba4d43" containerID="50c8080b3430dbbe269753f9b5a57dc077e3d33936f45f30c9ec89b77288c34b" exitCode=0 Jul 10 18:59:04.469431 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.469292 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" Jul 10 18:59:04.469431 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.469321 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" event={"ID":"f48768bb-db7c-4c0b-8869-dd11d8ba4d43","Type":"ContainerDied","Data":"50c8080b3430dbbe269753f9b5a57dc077e3d33936f45f30c9ec89b77288c34b"} Jul 10 18:59:04.469431 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.469369 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw" event={"ID":"f48768bb-db7c-4c0b-8869-dd11d8ba4d43","Type":"ContainerDied","Data":"f2e4846e001c06d9a7159a735d59c95dc51d65750f452ea285532a04985e3f0f"} Jul 10 18:59:04.469431 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.469387 2574 scope.go:117] "RemoveContainer" containerID="50c8080b3430dbbe269753f9b5a57dc077e3d33936f45f30c9ec89b77288c34b" Jul 10 18:59:04.472057 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.472022 2574 generic.go:356] "Generic (PLEG): container finished" podID="edeac1b8-36da-41de-9dbf-5e4424e5bac0" containerID="d6ba4c57ff01622320b63854786e7829ae4ba443aba3b9e121aedb62b3eba81e" exitCode=0 Jul 10 18:59:04.472183 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.472082 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7669-139946653555568" event={"ID":"edeac1b8-36da-41de-9dbf-5e4424e5bac0","Type":"ContainerDied","Data":"d6ba4c57ff01622320b63854786e7829ae4ba443aba3b9e121aedb62b3eba81e"} Jul 10 18:59:04.479214 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.479199 2574 scope.go:117] "RemoveContainer" containerID="50c8080b3430dbbe269753f9b5a57dc077e3d33936f45f30c9ec89b77288c34b" Jul 10 18:59:04.479495 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:59:04.479474 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50c8080b3430dbbe269753f9b5a57dc077e3d33936f45f30c9ec89b77288c34b\": container with ID starting with 50c8080b3430dbbe269753f9b5a57dc077e3d33936f45f30c9ec89b77288c34b not found: ID does not exist" containerID="50c8080b3430dbbe269753f9b5a57dc077e3d33936f45f30c9ec89b77288c34b" Jul 10 18:59:04.479563 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.479501 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50c8080b3430dbbe269753f9b5a57dc077e3d33936f45f30c9ec89b77288c34b"} err="failed to get container status \"50c8080b3430dbbe269753f9b5a57dc077e3d33936f45f30c9ec89b77288c34b\": rpc error: code = NotFound desc = could not find container \"50c8080b3430dbbe269753f9b5a57dc077e3d33936f45f30c9ec89b77288c34b\": container with ID starting with 50c8080b3430dbbe269753f9b5a57dc077e3d33936f45f30c9ec89b77288c34b not found: ID does not exist" Jul 10 18:59:04.490269 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.490248 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-tmp-dir\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:04.490269 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.490270 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-tls-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:04.490417 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.490286 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bvct\" (UniqueName: \"kubernetes.io/projected/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-kube-api-access-9bvct\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:04.490417 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.490300 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f48768bb-db7c-4c0b-8869-dd11d8ba4d43-dshm\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:04.503346 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.503315 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw"] Jul 10 18:59:04.508325 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:04.508306 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/rate-test-model-afdf1e-kserve-7c5ff9f8d7-cbcgw"] Jul 10 18:59:05.112737 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.112676 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" podUID="1930ec1c-3f14-4764-8581-f3e09fe76ad7" containerName="main" containerID="cri-o://027491ac816b5ba7b3b00db61517a9a4933f47110ec2f453ec0b57725fec2e1e" gracePeriod=45 Jul 10 18:59:05.316643 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.316606 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f48768bb-db7c-4c0b-8869-dd11d8ba4d43" path="/var/lib/kubelet/pods/f48768bb-db7c-4c0b-8869-dd11d8ba4d43/volumes" Jul 10 18:59:05.375509 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.375453 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:59:05.399502 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.399471 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-dshm\") pod \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " Jul 10 18:59:05.399664 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.399523 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7s8f\" (UniqueName: \"kubernetes.io/projected/1930ec1c-3f14-4764-8581-f3e09fe76ad7-kube-api-access-t7s8f\") pod \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " Jul 10 18:59:05.399664 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.399555 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-tmp-dir\") pod \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " Jul 10 18:59:05.399664 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.399640 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-model-cache\") pod \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " Jul 10 18:59:05.399834 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.399683 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-home\") pod \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " Jul 10 18:59:05.399834 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.399726 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1930ec1c-3f14-4764-8581-f3e09fe76ad7-tls-certs\") pod \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\" (UID: \"1930ec1c-3f14-4764-8581-f3e09fe76ad7\") " Jul 10 18:59:05.399948 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.399892 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-model-cache" (OuterVolumeSpecName: "model-cache") pod "1930ec1c-3f14-4764-8581-f3e09fe76ad7" (UID: "1930ec1c-3f14-4764-8581-f3e09fe76ad7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:05.400036 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.399899 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1930ec1c-3f14-4764-8581-f3e09fe76ad7" (UID: "1930ec1c-3f14-4764-8581-f3e09fe76ad7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:05.400115 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.400041 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-home" (OuterVolumeSpecName: "home") pod "1930ec1c-3f14-4764-8581-f3e09fe76ad7" (UID: "1930ec1c-3f14-4764-8581-f3e09fe76ad7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:05.400115 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.400076 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-model-cache\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:05.401823 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.401797 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1930ec1c-3f14-4764-8581-f3e09fe76ad7-kube-api-access-t7s8f" (OuterVolumeSpecName: "kube-api-access-t7s8f") pod "1930ec1c-3f14-4764-8581-f3e09fe76ad7" (UID: "1930ec1c-3f14-4764-8581-f3e09fe76ad7"). InnerVolumeSpecName "kube-api-access-t7s8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:59:05.401940 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.401843 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1930ec1c-3f14-4764-8581-f3e09fe76ad7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1930ec1c-3f14-4764-8581-f3e09fe76ad7" (UID: "1930ec1c-3f14-4764-8581-f3e09fe76ad7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:59:05.402170 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.402152 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-dshm" (OuterVolumeSpecName: "dshm") pod "1930ec1c-3f14-4764-8581-f3e09fe76ad7" (UID: "1930ec1c-3f14-4764-8581-f3e09fe76ad7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:05.477829 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.477793 2574 generic.go:356] "Generic (PLEG): container finished" podID="1930ec1c-3f14-4764-8581-f3e09fe76ad7" containerID="027491ac816b5ba7b3b00db61517a9a4933f47110ec2f453ec0b57725fec2e1e" exitCode=0 Jul 10 18:59:05.478035 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.477840 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" Jul 10 18:59:05.478035 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.477887 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" event={"ID":"1930ec1c-3f14-4764-8581-f3e09fe76ad7","Type":"ContainerDied","Data":"027491ac816b5ba7b3b00db61517a9a4933f47110ec2f453ec0b57725fec2e1e"} Jul 10 18:59:05.478035 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.477930 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl" event={"ID":"1930ec1c-3f14-4764-8581-f3e09fe76ad7","Type":"ContainerDied","Data":"86a8fbdfd5b59d3b04efccb4e0896101d20b3024884daf8a786a836ff02dbf90"} Jul 10 18:59:05.478035 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.477951 2574 scope.go:117] "RemoveContainer" containerID="027491ac816b5ba7b3b00db61517a9a4933f47110ec2f453ec0b57725fec2e1e" Jul 10 18:59:05.487205 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.487186 2574 scope.go:117] "RemoveContainer" containerID="027491ac816b5ba7b3b00db61517a9a4933f47110ec2f453ec0b57725fec2e1e" Jul 10 18:59:05.487525 ip-10-0-132-151 kubenswrapper[2574]: E0710 18:59:05.487506 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"027491ac816b5ba7b3b00db61517a9a4933f47110ec2f453ec0b57725fec2e1e\": container with ID starting with 027491ac816b5ba7b3b00db61517a9a4933f47110ec2f453ec0b57725fec2e1e not found: ID does not exist" containerID="027491ac816b5ba7b3b00db61517a9a4933f47110ec2f453ec0b57725fec2e1e" Jul 10 18:59:05.487579 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.487533 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"027491ac816b5ba7b3b00db61517a9a4933f47110ec2f453ec0b57725fec2e1e"} err="failed to get container status \"027491ac816b5ba7b3b00db61517a9a4933f47110ec2f453ec0b57725fec2e1e\": rpc error: code = NotFound desc = could not find container \"027491ac816b5ba7b3b00db61517a9a4933f47110ec2f453ec0b57725fec2e1e\": container with ID starting with 027491ac816b5ba7b3b00db61517a9a4933f47110ec2f453ec0b57725fec2e1e not found: ID does not exist" Jul 10 18:59:05.499529 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.499492 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl"] Jul 10 18:59:05.500729 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.500705 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-home\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:05.500729 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.500728 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1930ec1c-3f14-4764-8581-f3e09fe76ad7-tls-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:05.500905 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.500736 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-dshm\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:05.500905 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.500745 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7s8f\" (UniqueName: \"kubernetes.io/projected/1930ec1c-3f14-4764-8581-f3e09fe76ad7-kube-api-access-t7s8f\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:05.500905 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.500754 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1930ec1c-3f14-4764-8581-f3e09fe76ad7-tmp-dir\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:05.503919 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.503894 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f83faf/rate-test-model-f83faf-kserve-8985fc4c9-jw7gl"] Jul 10 18:59:05.625232 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.625208 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653555568" Jul 10 18:59:05.651996 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.651919 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7669-139946653555568_edeac1b8-36da-41de-9dbf-5e4424e5bac0/test-curl-7669-139946653555568/0.log" Jul 10 18:59:05.682250 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.682216 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7669-139946653555568"] Jul 10 18:59:05.687865 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.687840 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7669-139946653555568"] Jul 10 18:59:05.703023 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.702995 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-447h9\" (UniqueName: \"kubernetes.io/projected/edeac1b8-36da-41de-9dbf-5e4424e5bac0-kube-api-access-447h9\") pod \"edeac1b8-36da-41de-9dbf-5e4424e5bac0\" (UID: \"edeac1b8-36da-41de-9dbf-5e4424e5bac0\") " Jul 10 18:59:05.705132 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.705087 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edeac1b8-36da-41de-9dbf-5e4424e5bac0-kube-api-access-447h9" (OuterVolumeSpecName: "kube-api-access-447h9") pod "edeac1b8-36da-41de-9dbf-5e4424e5bac0" (UID: "edeac1b8-36da-41de-9dbf-5e4424e5bac0"). InnerVolumeSpecName "kube-api-access-447h9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:59:05.803958 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.803918 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-447h9\" (UniqueName: \"kubernetes.io/projected/edeac1b8-36da-41de-9dbf-5e4424e5bac0-kube-api-access-447h9\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:05.820258 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.820219 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7669-139946653555568"] Jul 10 18:59:05.820677 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.820657 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edeac1b8-36da-41de-9dbf-5e4424e5bac0" containerName="test-curl-7669-139946653555568" Jul 10 18:59:05.820771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.820679 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="edeac1b8-36da-41de-9dbf-5e4424e5bac0" containerName="test-curl-7669-139946653555568" Jul 10 18:59:05.820771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.820710 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f48768bb-db7c-4c0b-8869-dd11d8ba4d43" containerName="main" Jul 10 18:59:05.820771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.820720 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f48768bb-db7c-4c0b-8869-dd11d8ba4d43" containerName="main" Jul 10 18:59:05.820771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.820737 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1930ec1c-3f14-4764-8581-f3e09fe76ad7" containerName="main" Jul 10 18:59:05.820771 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.820745 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1930ec1c-3f14-4764-8581-f3e09fe76ad7" containerName="main" Jul 10 18:59:05.821002 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.820852 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="edeac1b8-36da-41de-9dbf-5e4424e5bac0" containerName="test-curl-7669-139946653555568" Jul 10 18:59:05.821002 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.820867 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f48768bb-db7c-4c0b-8869-dd11d8ba4d43" containerName="main" Jul 10 18:59:05.821002 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.820879 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1930ec1c-3f14-4764-8581-f3e09fe76ad7" containerName="main" Jul 10 18:59:05.823873 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.823852 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653555568" Jul 10 18:59:05.833638 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.833612 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7669-139946653555568"] Jul 10 18:59:05.905157 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:05.905036 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7bss\" (UniqueName: \"kubernetes.io/projected/070fe947-6592-43d4-891a-b487c585b204-kube-api-access-j7bss\") pod \"test-curl-7669-139946653555568\" (UID: \"070fe947-6592-43d4-891a-b487c585b204\") " pod="openshift-ingress/test-curl-7669-139946653555568" Jul 10 18:59:06.006454 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:06.006402 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7bss\" (UniqueName: \"kubernetes.io/projected/070fe947-6592-43d4-891a-b487c585b204-kube-api-access-j7bss\") pod \"test-curl-7669-139946653555568\" (UID: \"070fe947-6592-43d4-891a-b487c585b204\") " pod="openshift-ingress/test-curl-7669-139946653555568" Jul 10 18:59:06.015498 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:06.015464 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7bss\" (UniqueName: \"kubernetes.io/projected/070fe947-6592-43d4-891a-b487c585b204-kube-api-access-j7bss\") pod \"test-curl-7669-139946653555568\" (UID: \"070fe947-6592-43d4-891a-b487c585b204\") " pod="openshift-ingress/test-curl-7669-139946653555568" Jul 10 18:59:06.133236 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:06.133199 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653555568" Jul 10 18:59:06.266777 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:06.266754 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7669-139946653555568"] Jul 10 18:59:06.268427 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:59:06.268399 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod070fe947_6592_43d4_891a_b487c585b204.slice/crio-93840eda7bf2c7c015c49b8b802ec3558bfc4f23caa1ba253e1c28454e068c69 WatchSource:0}: Error finding container 93840eda7bf2c7c015c49b8b802ec3558bfc4f23caa1ba253e1c28454e068c69: Status 404 returned error can't find the container with id 93840eda7bf2c7c015c49b8b802ec3558bfc4f23caa1ba253e1c28454e068c69 Jul 10 18:59:06.485219 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:06.485119 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7669-139946653555568" event={"ID":"070fe947-6592-43d4-891a-b487c585b204","Type":"ContainerStarted","Data":"93840eda7bf2c7c015c49b8b802ec3558bfc4f23caa1ba253e1c28454e068c69"} Jul 10 18:59:06.486375 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:06.486346 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab56babc0460b5733b78592acbc604c72bc08105677bff0632cde0de0aec9c8e" Jul 10 18:59:06.486375 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:06.486352 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653555568" Jul 10 18:59:06.497980 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:06.497947 2574 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7669-139946653555568" oldPodUID="edeac1b8-36da-41de-9dbf-5e4424e5bac0" podUID="070fe947-6592-43d4-891a-b487c585b204" Jul 10 18:59:07.315446 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:07.315413 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1930ec1c-3f14-4764-8581-f3e09fe76ad7" path="/var/lib/kubelet/pods/1930ec1c-3f14-4764-8581-f3e09fe76ad7/volumes" Jul 10 18:59:07.315950 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:07.315899 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edeac1b8-36da-41de-9dbf-5e4424e5bac0" path="/var/lib/kubelet/pods/edeac1b8-36da-41de-9dbf-5e4424e5bac0/volumes" Jul 10 18:59:07.493749 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:07.493714 2574 generic.go:356] "Generic (PLEG): container finished" podID="070fe947-6592-43d4-891a-b487c585b204" containerID="b291ef1787e33e4befc6b4d5ac490a0e4afa2ec35d2dab4e26ef30653c96d22e" exitCode=0 Jul 10 18:59:07.493926 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:07.493754 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7669-139946653555568" event={"ID":"070fe947-6592-43d4-891a-b487c585b204","Type":"ContainerDied","Data":"b291ef1787e33e4befc6b4d5ac490a0e4afa2ec35d2dab4e26ef30653c96d22e"} Jul 10 18:59:08.647687 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:08.647664 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653555568" Jul 10 18:59:08.679647 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:08.679623 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7669-139946653555568_070fe947-6592-43d4-891a-b487c585b204/test-curl-7669-139946653555568/0.log" Jul 10 18:59:08.708436 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:08.708401 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7669-139946653555568"] Jul 10 18:59:08.713015 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:08.712991 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7669-139946653555568"] Jul 10 18:59:08.736341 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:08.736321 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7bss\" (UniqueName: \"kubernetes.io/projected/070fe947-6592-43d4-891a-b487c585b204-kube-api-access-j7bss\") pod \"070fe947-6592-43d4-891a-b487c585b204\" (UID: \"070fe947-6592-43d4-891a-b487c585b204\") " Jul 10 18:59:08.738454 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:08.738402 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/070fe947-6592-43d4-891a-b487c585b204-kube-api-access-j7bss" (OuterVolumeSpecName: "kube-api-access-j7bss") pod "070fe947-6592-43d4-891a-b487c585b204" (UID: "070fe947-6592-43d4-891a-b487c585b204"). InnerVolumeSpecName "kube-api-access-j7bss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:59:08.837174 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:08.837065 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7bss\" (UniqueName: \"kubernetes.io/projected/070fe947-6592-43d4-891a-b487c585b204-kube-api-access-j7bss\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 10 18:59:08.998257 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:08.998216 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:09.315759 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:09.315726 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="070fe947-6592-43d4-891a-b487c585b204" path="/var/lib/kubelet/pods/070fe947-6592-43d4-891a-b487c585b204/volumes" Jul 10 18:59:09.505737 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:09.505703 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653555568" Jul 10 18:59:09.505907 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:09.505710 2574 scope.go:117] "RemoveContainer" containerID="b291ef1787e33e4befc6b4d5ac490a0e4afa2ec35d2dab4e26ef30653c96d22e" Jul 10 18:59:09.551403 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:09.551353 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:09.816279 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:09.816236 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:09.915905 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:09.915870 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:10.206570 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:10.206530 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:14.282121 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:14.281996 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:59:16.144173 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:16.144141 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:17.368734 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:17.368696 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:17.470513 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:17.470479 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:17.612031 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:17.611996 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:23.365193 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:23.365147 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:23.467979 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:23.467946 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:23.672409 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:23.672327 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:23.920171 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:23.920123 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:26.880015 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:26.879983 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgmjb"] Jul 10 18:59:30.159861 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:30.159832 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d75465dc8-9xgl4_003942ce-485d-48ee-9261-5f72541c940f/copy-manifests/0.log" Jul 10 18:59:30.168056 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:30.168034 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d75465dc8-9xgl4_003942ce-485d-48ee-9261-5f72541c940f/manager/0.log" Jul 10 18:59:32.906079 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:32.906047 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-tgmjb_ffb6f688-9ecd-4c59-bdcc-707407d60eb2/limitador/0.log" Jul 10 18:59:33.637076 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:33.637045 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79bb7c45b8-qtp99_445ffa19-7b40-4f32-8e19-fb30f8f045bd/kube-auth-proxy/0.log" Jul 10 18:59:33.992112 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:33.992061 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8b5nf_6ffce838-f116-412b-a24e-87fce044078e/payload-pre-processing/0.log" Jul 10 18:59:34.260416 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:34.260344 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57f678b4d6-q2rhg_b8a376b5-b2dc-439a-aa9c-b1bd4f37252d/router/0.log" Jul 10 18:59:34.967723 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:34.967694 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbftwf6_8f5a9698-95df-4831-9c0e-daa4c7b29ce9/main/0.log" Jul 10 18:59:35.219089 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:35.219010 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-p5kzl_ed4743f8-b7fa-4e73-bd30-5fa474bef07c/main/0.log" Jul 10 18:59:39.225543 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:39.225504 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:39.322821 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:39.322786 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" limit=1024 Jul 10 18:59:42.143320 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:42.143291 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-84n7b_97b02d51-cfd3-4d11-a108-c54bf9dca12e/global-pull-secret-syncer/0.log" Jul 10 18:59:42.305038 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:42.305013 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9rhl2_952ce2e7-c21f-4fe0-9357-ced0cedef546/konnectivity-agent/0.log" Jul 10 18:59:42.403922 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:42.403845 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-151.ec2.internal_7554e05d30e68919ee44413a4c1cb32c/haproxy/0.log" Jul 10 18:59:48.756072 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:48.756042 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-tgmjb_ffb6f688-9ecd-4c59-bdcc-707407d60eb2/limitador/0.log" Jul 10 18:59:50.457972 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:50.457941 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_229c3dcc-b6f5-4c24-8230-cd427d420cbe/alertmanager/0.log" Jul 10 18:59:50.485265 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:50.485221 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_229c3dcc-b6f5-4c24-8230-cd427d420cbe/config-reloader/0.log" Jul 10 18:59:50.512987 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:50.512961 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_229c3dcc-b6f5-4c24-8230-cd427d420cbe/kube-rbac-proxy-web/0.log" Jul 10 18:59:50.538758 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:50.538736 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_229c3dcc-b6f5-4c24-8230-cd427d420cbe/kube-rbac-proxy/0.log" Jul 10 18:59:50.562894 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:50.562865 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_229c3dcc-b6f5-4c24-8230-cd427d420cbe/kube-rbac-proxy-metric/0.log" Jul 10 18:59:50.587763 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:50.587743 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_229c3dcc-b6f5-4c24-8230-cd427d420cbe/prom-label-proxy/0.log" Jul 10 18:59:50.617013 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:50.616988 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_229c3dcc-b6f5-4c24-8230-cd427d420cbe/init-config-reloader/0.log" Jul 10 18:59:50.661274 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:50.661243 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-scpbz_20f79a38-b179-4205-bf87-71983574bfdd/cluster-monitoring-operator/0.log" Jul 10 18:59:50.703033 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:50.702997 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-lxj76_8e821829-aa45-49a3-a7ef-676c90c57ac2/kube-state-metrics/0.log" Jul 10 18:59:50.725876 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:50.725803 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-lxj76_8e821829-aa45-49a3-a7ef-676c90c57ac2/kube-rbac-proxy-main/0.log" Jul 10 18:59:50.750884 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:50.750856 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-lxj76_8e821829-aa45-49a3-a7ef-676c90c57ac2/kube-rbac-proxy-self/0.log" Jul 10 18:59:50.809406 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:50.809374 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-6r5zz_82fa2512-54af-40f6-9e11-baa115a0bb50/monitoring-plugin/0.log" Jul 10 18:59:51.030546 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:51.030457 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sh4hv_d1b45de6-e9ef-409d-82f9-80ec78ffdd27/node-exporter/0.log" Jul 10 18:59:51.054975 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:51.054948 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sh4hv_d1b45de6-e9ef-409d-82f9-80ec78ffdd27/kube-rbac-proxy/0.log" Jul 10 18:59:51.078663 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:51.078640 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sh4hv_d1b45de6-e9ef-409d-82f9-80ec78ffdd27/init-textfile/0.log" Jul 10 18:59:51.213640 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:51.213612 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_531c296a-3a99-47dd-91dc-fa754f1523f4/prometheus/0.log" Jul 10 18:59:51.239031 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:51.238996 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_531c296a-3a99-47dd-91dc-fa754f1523f4/config-reloader/0.log" Jul 10 18:59:51.263266 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:51.263225 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_531c296a-3a99-47dd-91dc-fa754f1523f4/thanos-sidecar/0.log" Jul 10 18:59:51.289577 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:51.289484 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_531c296a-3a99-47dd-91dc-fa754f1523f4/kube-rbac-proxy-web/0.log" Jul 10 18:59:51.314867 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:51.314844 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_531c296a-3a99-47dd-91dc-fa754f1523f4/kube-rbac-proxy/0.log" Jul 10 18:59:51.339520 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:51.339497 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_531c296a-3a99-47dd-91dc-fa754f1523f4/kube-rbac-proxy-thanos/0.log" Jul 10 18:59:51.364451 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:51.364425 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_531c296a-3a99-47dd-91dc-fa754f1523f4/init-config-reloader/0.log" Jul 10 18:59:51.494677 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:51.494648 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d56fbf68d-9qzzb_56a7f6c5-3b0f-417e-8acf-cc12d9871a6f/telemeter-client/0.log" Jul 10 18:59:51.531548 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:51.531524 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d56fbf68d-9qzzb_56a7f6c5-3b0f-417e-8acf-cc12d9871a6f/reload/0.log" Jul 10 18:59:51.560713 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:51.560621 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d56fbf68d-9qzzb_56a7f6c5-3b0f-417e-8acf-cc12d9871a6f/kube-rbac-proxy/0.log" Jul 10 18:59:52.122299 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.122265 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp"] Jul 10 18:59:52.122649 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.122635 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="070fe947-6592-43d4-891a-b487c585b204" containerName="test-curl-7669-139946653555568" Jul 10 18:59:52.122649 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.122650 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="070fe947-6592-43d4-891a-b487c585b204" containerName="test-curl-7669-139946653555568" Jul 10 18:59:52.122737 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.122719 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="070fe947-6592-43d4-891a-b487c585b204" containerName="test-curl-7669-139946653555568" Jul 10 18:59:52.126256 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.126234 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.128716 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.128694 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9xwdk\"/\"kube-root-ca.crt\"" Jul 10 18:59:52.129629 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.129610 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9xwdk\"/\"openshift-service-ca.crt\"" Jul 10 18:59:52.129745 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.129686 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9xwdk\"/\"default-dockercfg-rjp4f\"" Jul 10 18:59:52.137510 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.137488 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp"] Jul 10 18:59:52.247143 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.247107 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f91e62a-2dce-4ded-abe5-628b16e66ff9-sys\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.247313 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.247149 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f91e62a-2dce-4ded-abe5-628b16e66ff9-lib-modules\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.247313 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.247253 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f91e62a-2dce-4ded-abe5-628b16e66ff9-proc\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.247392 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.247314 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f91e62a-2dce-4ded-abe5-628b16e66ff9-podres\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.247392 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.247373 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j49kl\" (UniqueName: \"kubernetes.io/projected/5f91e62a-2dce-4ded-abe5-628b16e66ff9-kube-api-access-j49kl\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.348060 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.348020 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f91e62a-2dce-4ded-abe5-628b16e66ff9-sys\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.348060 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.348059 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f91e62a-2dce-4ded-abe5-628b16e66ff9-lib-modules\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.348339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.348131 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f91e62a-2dce-4ded-abe5-628b16e66ff9-proc\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.348339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.348176 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f91e62a-2dce-4ded-abe5-628b16e66ff9-podres\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.348339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.348175 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f91e62a-2dce-4ded-abe5-628b16e66ff9-sys\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.348339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.348250 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j49kl\" (UniqueName: \"kubernetes.io/projected/5f91e62a-2dce-4ded-abe5-628b16e66ff9-kube-api-access-j49kl\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.348339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.348252 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f91e62a-2dce-4ded-abe5-628b16e66ff9-proc\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.348339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.348271 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f91e62a-2dce-4ded-abe5-628b16e66ff9-lib-modules\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.348339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.348329 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f91e62a-2dce-4ded-abe5-628b16e66ff9-podres\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.356857 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.356824 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j49kl\" (UniqueName: \"kubernetes.io/projected/5f91e62a-2dce-4ded-abe5-628b16e66ff9-kube-api-access-j49kl\") pod \"perf-node-gather-daemonset-rzxzp\" (UID: \"5f91e62a-2dce-4ded-abe5-628b16e66ff9\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.437078 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.437045 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.574402 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.574333 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp"] Jul 10 18:59:52.577780 ip-10-0-132-151 kubenswrapper[2574]: W0710 18:59:52.577746 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5f91e62a_2dce_4ded_abe5_628b16e66ff9.slice/crio-29744140b08992987e263a5218630804404151b17cc5e620e24549840eed0bfa WatchSource:0}: Error finding container 29744140b08992987e263a5218630804404151b17cc5e620e24549840eed0bfa: Status 404 returned error can't find the container with id 29744140b08992987e263a5218630804404151b17cc5e620e24549840eed0bfa Jul 10 18:59:52.720680 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.720584 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" event={"ID":"5f91e62a-2dce-4ded-abe5-628b16e66ff9","Type":"ContainerStarted","Data":"078fd91d2bb7e56816e9d48385e1571acb976909452f323eaec48f488f6ec716"} Jul 10 18:59:52.720680 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.720636 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" event={"ID":"5f91e62a-2dce-4ded-abe5-628b16e66ff9","Type":"ContainerStarted","Data":"29744140b08992987e263a5218630804404151b17cc5e620e24549840eed0bfa"} Jul 10 18:59:52.720680 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.720663 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:52.754304 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:52.754253 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" podStartSLOduration=0.754237656 podStartE2EDuration="754.237656ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:59:52.753083373 +0000 UTC m=+2552.166368159" watchObservedRunningTime="2026-07-10 18:59:52.754237656 +0000 UTC m=+2552.167522442" Jul 10 18:59:53.550916 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:53.550882 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/2.log" Jul 10 18:59:53.555129 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:53.555087 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-cjjrj_e3770e0a-c127-454b-b0dc-6e4d4a9c1628/console-operator/3.log" Jul 10 18:59:54.700297 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:54.700242 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-vp5ls_e953a150-6569-49ea-8f29-fc6503cf328a/volume-data-source-validator/0.log" Jul 10 18:59:55.752987 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:55.752960 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7nprf_d29817f3-db17-4d2b-9666-20e677724b75/dns/0.log" Jul 10 18:59:55.778637 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:55.778612 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7nprf_d29817f3-db17-4d2b-9666-20e677724b75/kube-rbac-proxy/0.log" Jul 10 18:59:55.943717 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:55.943683 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7qhmx_42571c86-4eaf-44a2-868e-1e023a349549/dns-node-resolver/0.log" Jul 10 18:59:56.614785 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:56.614759 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c8b4p_1e02735c-c7f7-4070-9a1f-0410bace9cd9/node-ca/0.log" Jul 10 18:59:57.843701 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:57.843672 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79bb7c45b8-qtp99_445ffa19-7b40-4f32-8e19-fb30f8f045bd/kube-auth-proxy/0.log" Jul 10 18:59:57.995019 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:57.994990 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8b5nf_6ffce838-f116-412b-a24e-87fce044078e/payload-pre-processing/0.log" Jul 10 18:59:58.052118 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:58.052069 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57f678b4d6-q2rhg_b8a376b5-b2dc-439a-aa9c-b1bd4f37252d/router/0.log" Jul 10 18:59:58.735908 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:58.735883 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-rzxzp" Jul 10 18:59:58.756955 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:58.756923 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cpmcx_60217543-943f-4011-8a09-132f51157b28/serve-healthcheck-canary/0.log" Jul 10 18:59:59.326038 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:59.326004 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-b5h8k_06f82976-1aae-4012-bac5-792e61e2c8af/insights-operator/0.log" Jul 10 18:59:59.327339 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:59.327322 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-b5h8k_06f82976-1aae-4012-bac5-792e61e2c8af/insights-operator/1.log" Jul 10 18:59:59.350500 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:59.350477 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4n9wp_0533d63c-4951-4049-9f06-119eda06b602/kube-rbac-proxy/0.log" Jul 10 18:59:59.376058 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:59.376029 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4n9wp_0533d63c-4951-4049-9f06-119eda06b602/exporter/0.log" Jul 10 18:59:59.403347 ip-10-0-132-151 kubenswrapper[2574]: I0710 18:59:59.403322 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4n9wp_0533d63c-4951-4049-9f06-119eda06b602/extractor/0.log" Jul 10 19:00:02.613255 ip-10-0-132-151 kubenswrapper[2574]: I0710 19:00:02.613227 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d75465dc8-9xgl4_003942ce-485d-48ee-9261-5f72541c940f/manager/0.log" Jul 10 19:00:02.639636 ip-10-0-132-151 kubenswrapper[2574]: I0710 19:00:02.639607 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d75465dc8-9xgl4_003942ce-485d-48ee-9261-5f72541c940f/copy-manifests/0.log" Jul 10 19:00:03.426849 ip-10-0-132-151 kubenswrapper[2574]: I0710 19:00:03.426820 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79bb7c45b8-qtp99_445ffa19-7b40-4f32-8e19-fb30f8f045bd/kube-auth-proxy/0.log" Jul 10 19:00:03.531279 ip-10-0-132-151 kubenswrapper[2574]: I0710 19:00:03.531246 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8b5nf_6ffce838-f116-412b-a24e-87fce044078e/payload-pre-processing/0.log" Jul 10 19:00:03.569074 ip-10-0-132-151 kubenswrapper[2574]: I0710 19:00:03.569042 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57f678b4d6-q2rhg_b8a376b5-b2dc-439a-aa9c-b1bd4f37252d/router/0.log" Jul 10 19:00:04.115831 ip-10-0-132-151 kubenswrapper[2574]: I0710 19:00:04.115807 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6cfc4d774-stv4n_53b8d0b4-91d8-4e6d-a74c-7342d69edd26/manager/0.log" Jul 10 19:00:04.155337 ip-10-0-132-151 kubenswrapper[2574]: I0710 19:00:04.155310 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-kcm79_24035f6b-02a5-4d95-afbb-a8e1bf528334/openshift-lws-operator/0.log" Jul 10 19:00:04.931967 ip-10-0-132-151 kubenswrapper[2574]: I0710 19:00:04.931943 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79bb7c45b8-qtp99_445ffa19-7b40-4f32-8e19-fb30f8f045bd/kube-auth-proxy/0.log" Jul 10 19:00:05.036653 ip-10-0-132-151 kubenswrapper[2574]: I0710 19:00:05.036625 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8b5nf_6ffce838-f116-412b-a24e-87fce044078e/payload-pre-processing/0.log" Jul 10 19:00:05.076658 ip-10-0-132-151 kubenswrapper[2574]: I0710 19:00:05.076629 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57f678b4d6-q2rhg_b8a376b5-b2dc-439a-aa9c-b1bd4f37252d/router/0.log" Jul 10 19:00:09.535295 ip-10-0-132-151 kubenswrapper[2574]: I0710 19:00:09.535261 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-7x8dq_d6ade376-624c-4dfc-858b-1940181dfbb3/kube-storage-version-migrator-operator/1.log" Jul 10 19:00:09.536136 ip-10-0-132-151 kubenswrapper[2574]: I0710 19:00:09.536118 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-7x8dq_d6ade376-624c-4dfc-858b-1940181dfbb3/kube-storage-version-migrator-operator/0.log"