Jul 07 17:39:55.827448 ip-10-0-129-170 systemd[1]: Starting Kubernetes Kubelet... Jul 07 17:39:56.295886 ip-10-0-129-170 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 17:39:56.295886 ip-10-0-129-170 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 17:39:56.295886 ip-10-0-129-170 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 17:39:56.295886 ip-10-0-129-170 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 17:39:56.295886 ip-10-0-129-170 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 17:39:56.296985 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.296886 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 17:39:56.304280 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304249 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:39:56.304280 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304276 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:39:56.304280 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304281 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:39:56.304280 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304284 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304288 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304290 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304293 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304296 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304299 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304301 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304304 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304306 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304309 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304312 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304320 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304323 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304326 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304329 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304331 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304333 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304336 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304338 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304341 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:39:56.304416 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304343 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304346 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304349 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304351 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304354 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304356 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304358 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304361 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304363 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304365 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304367 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304370 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304372 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304375 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304378 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304381 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304384 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304386 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304389 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304391 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:39:56.304924 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304393 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304396 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304399 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304402 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304404 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304407 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304409 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304412 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304414 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304417 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304419 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304422 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304424 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304427 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304430 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304435 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304440 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304442 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304445 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304448 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:39:56.305483 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304451 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304455 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304460 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304464 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304467 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304470 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304473 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304476 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304479 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304483 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304485 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304488 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304490 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304493 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304495 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304498 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304500 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304502 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304505 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:39:56.305979 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304508 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304512 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304514 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304517 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304521 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304523 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304526 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304528 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304531 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304533 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304535 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304538 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304540 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304542 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304544 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304547 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.304550 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305456 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305470 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305473 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:39:56.306452 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305476 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305479 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305483 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305486 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305489 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305493 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305496 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305498 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305501 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305504 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305506 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305510 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305515 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305518 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305521 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305524 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305526 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305529 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305531 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:39:56.306931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305534 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305537 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305541 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305544 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305546 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305549 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305551 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305554 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305556 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305559 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305562 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305572 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305574 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305577 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305581 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305585 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305589 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305591 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305595 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:39:56.307426 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305600 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305603 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305606 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305609 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305611 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305614 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305616 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305619 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305621 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305624 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305627 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305629 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305631 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305634 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305637 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305639 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305642 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305644 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305648 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305651 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:39:56.307935 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305653 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305656 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305658 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305662 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305665 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305668 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305671 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305674 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305677 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305680 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305682 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305685 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305687 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305689 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305692 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305694 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305697 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305699 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305702 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:39:56.308448 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305704 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305706 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305709 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305711 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305714 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305716 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305719 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305721 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305723 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305726 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305728 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305730 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305733 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305735 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305738 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305740 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305743 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305746 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.305748 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306180 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:39:56.308936 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306188 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306191 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306193 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306196 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306199 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306201 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306204 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306207 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306210 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306212 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306215 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306218 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306220 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306223 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306226 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306229 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306231 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306234 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306237 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:39:56.309460 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306239 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306241 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306245 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306247 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306250 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306253 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306256 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306258 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306261 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306264 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306267 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306272 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306276 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306279 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306281 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306284 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306287 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306289 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306292 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306295 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:39:56.309918 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306298 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306301 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306304 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306306 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306309 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306311 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306314 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306317 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306319 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306323 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306325 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306328 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306330 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306333 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306336 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306338 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306341 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306343 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306347 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:39:56.310516 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306351 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306354 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306358 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306361 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306364 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306366 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306370 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306372 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306375 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306377 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306380 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306382 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306385 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306387 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306390 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306392 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306396 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306398 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306401 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306403 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:39:56.310978 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306406 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306410 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306413 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306416 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306420 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306423 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306425 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306427 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306430 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306433 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306435 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306438 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306440 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306442 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306449 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306452 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306454 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306457 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306460 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.306463 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:39:56.311493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306550 2577 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306562 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306569 2577 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306573 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306578 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306581 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306586 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306590 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306593 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306597 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306601 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306604 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306607 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306610 2577 flags.go:64] FLAG: --cgroup-root="" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306613 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306616 2577 flags.go:64] FLAG: --client-ca-file="" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306618 2577 flags.go:64] FLAG: --cloud-provider="external" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306622 2577 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306626 2577 flags.go:64] FLAG: --cluster-domain="" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306629 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306633 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306636 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306639 2577 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306648 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 17:39:56.311999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306651 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306654 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306658 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306662 2577 flags.go:64] FLAG: --contention-profiling="false" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306665 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306668 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306671 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306674 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306679 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306682 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306685 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306688 2577 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306691 2577 flags.go:64] FLAG: --enable-server="true" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306694 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306699 2577 flags.go:64] FLAG: --event-burst="100" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306702 2577 flags.go:64] FLAG: --event-qps="50" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306705 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306708 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306710 2577 flags.go:64] FLAG: --eviction-hard="" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306714 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306716 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306719 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306722 2577 flags.go:64] FLAG: --eviction-soft="" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306725 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306728 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 17:39:56.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306731 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306734 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306737 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306740 2577 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306743 2577 flags.go:64] FLAG: --feature-gates="" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306747 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306750 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306753 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306756 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306760 2577 flags.go:64] FLAG: --healthz-port="10248" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306763 2577 flags.go:64] FLAG: --help="false" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306766 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306769 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306772 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306775 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306778 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306783 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306785 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306788 2577 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306792 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306795 2577 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306798 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306801 2577 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306804 2577 flags.go:64] FLAG: --kube-reserved="" Jul 07 17:39:56.313287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306807 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306810 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306813 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306816 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306819 2577 flags.go:64] FLAG: --lock-file="" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306822 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306825 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306828 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306833 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306836 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306840 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306843 2577 flags.go:64] FLAG: --logging-format="text" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306846 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306849 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306852 2577 flags.go:64] FLAG: --manifest-url="" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306855 2577 flags.go:64] FLAG: --manifest-url-header="" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306859 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306864 2577 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306868 2577 flags.go:64] FLAG: --max-pods="110" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306871 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306874 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306877 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306880 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306882 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306885 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 17:39:56.313880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306888 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306896 2577 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306899 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306903 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306906 2577 flags.go:64] FLAG: --pod-cidr="" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306909 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306914 2577 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306917 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306920 2577 flags.go:64] FLAG: --pods-per-core="0" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306922 2577 flags.go:64] FLAG: --port="10250" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306925 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306928 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00e6a266647ff2ebe" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306932 2577 flags.go:64] FLAG: --qos-reserved="" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306935 2577 flags.go:64] FLAG: --read-only-port="10255" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306939 2577 flags.go:64] FLAG: --register-node="true" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306942 2577 flags.go:64] FLAG: --register-with-taints="" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306946 2577 flags.go:64] FLAG: --registry-burst="10" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306951 2577 flags.go:64] FLAG: --registry-qps="5" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306953 2577 flags.go:64] FLAG: --reserved-cpus="" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306956 2577 flags.go:64] FLAG: --reserved-memory="" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306960 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306963 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306966 2577 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306969 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306972 2577 flags.go:64] FLAG: --runonce="false" Jul 07 17:39:56.314573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306976 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306979 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306982 2577 flags.go:64] FLAG: --seccomp-default="false" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306985 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306988 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306991 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306994 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.306998 2577 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307001 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307004 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307007 2577 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307010 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307013 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307015 2577 flags.go:64] FLAG: --system-cgroups="" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307018 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307023 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307026 2577 flags.go:64] FLAG: --tls-cert-file="" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307029 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307034 2577 flags.go:64] FLAG: --tls-min-version="" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307051 2577 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307054 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307057 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307060 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307063 2577 flags.go:64] FLAG: --v="2" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307070 2577 flags.go:64] FLAG: --version="false" Jul 07 17:39:56.315669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307074 2577 flags.go:64] FLAG: --vmodule="" Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307078 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.307082 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307230 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307235 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307238 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307241 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307247 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307251 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307253 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307256 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307259 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307262 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307265 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307267 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307270 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307273 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307275 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307278 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307281 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:39:56.316867 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307283 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307286 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307289 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307291 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307294 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307296 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307299 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307301 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307304 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307308 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307310 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307315 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307317 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307320 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307323 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307325 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307328 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307331 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307333 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307337 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:39:56.317757 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307340 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307344 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307347 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307350 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307353 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307355 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307358 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307360 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307363 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307366 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307369 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307371 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307374 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307376 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307379 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307381 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307384 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307387 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307389 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:39:56.318577 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307392 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307394 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307397 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307399 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307404 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307406 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307409 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307411 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307414 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307417 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307420 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307422 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307426 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307429 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307432 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307435 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307438 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307440 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307443 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:39:56.319130 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307445 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307448 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307450 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307455 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307460 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307463 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307466 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307468 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307472 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307475 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307478 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307481 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307484 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307486 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307490 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307492 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307495 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307499 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307502 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:39:56.319847 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307504 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307507 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307509 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307511 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.307514 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.308262 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.316893 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.316921 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.316951 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.316961 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317075 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317087 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317092 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317097 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317102 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:39:56.320673 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317106 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317110 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317115 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317118 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317126 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317132 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317137 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317142 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317146 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317150 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317155 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317159 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317163 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317167 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317171 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317175 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317179 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317184 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317189 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:39:56.321244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317193 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317197 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317202 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317206 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317211 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317215 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317220 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317223 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317228 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317232 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317238 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317242 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317246 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317250 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317255 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317258 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317263 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317267 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317271 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:39:56.321729 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317276 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317283 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317290 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317294 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317298 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317302 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317308 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317312 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317316 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317321 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317326 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317330 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317334 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317339 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317343 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317347 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317352 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317356 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317361 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317365 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:39:56.322339 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317370 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317376 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317382 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317386 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317392 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317396 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317402 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317406 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317410 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317414 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317418 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317422 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317426 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317431 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317434 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317439 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317443 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317447 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317451 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:39:56.322855 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317455 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317459 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317463 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317468 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317471 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317476 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317480 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317484 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317488 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317492 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317496 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317501 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317505 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317509 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317513 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317517 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:39:56.323382 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317522 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.317530 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317729 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317738 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317743 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317748 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317753 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317757 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317761 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317765 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317769 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317774 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317778 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317781 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317785 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 17:39:56.323842 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317789 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317793 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317797 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317801 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317808 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317813 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317817 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317822 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317828 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317832 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317837 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317841 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317846 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317851 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317856 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317860 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317864 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317869 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317873 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317877 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 17:39:56.324244 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317882 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317887 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317891 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317895 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317899 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317903 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317908 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317912 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317916 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317920 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317927 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317932 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317937 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317941 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317946 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317950 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317955 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317959 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317963 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 17:39:56.324810 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317968 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317973 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317977 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317981 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317985 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317989 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317994 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.317997 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318001 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318006 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318010 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318014 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318018 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318022 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318027 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318031 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318035 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318058 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318062 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318066 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 17:39:56.325368 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318070 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318074 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318078 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318082 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318086 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318091 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318095 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318099 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318104 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318108 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318112 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318117 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318121 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318126 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318130 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318134 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318139 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318143 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318147 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318151 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 17:39:56.325882 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318155 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 07 17:39:56.326397 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318159 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 17:39:56.326397 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318163 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 17:39:56.326397 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318167 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 17:39:56.326397 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318171 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 17:39:56.326397 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318175 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 17:39:56.326397 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:56.318181 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 17:39:56.326397 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.318191 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 17:39:56.326397 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.318390 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 17:39:56.326397 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.321813 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 17:39:56.326397 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.323628 2577 server.go:1019] "Starting client certificate rotation" Jul 07 17:39:56.326397 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.323755 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 17:39:56.326397 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.323803 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 17:39:56.327879 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.327855 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 17:39:56.338023 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.337986 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 17:39:56.346363 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.346336 2577 log.go:25] "Validated CRI v1 runtime API" Jul 07 17:39:56.350666 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.350642 2577 log.go:25] "Validated CRI v1 image API" Jul 07 17:39:56.351590 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.351558 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 17:39:56.354359 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.354331 2577 fs.go:135] Filesystem UUIDs: map[2ae67d6d-513d-4ea0-8fa5-f1ab491361d2:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9c330928-8a6d-4b7b-92b3-4947f50a4376:/dev/nvme0n1p3] Jul 07 17:39:56.354468 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.354355 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 17:39:56.360820 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.360667 2577 manager.go:216] Machine: {Timestamp:2026-07-07 17:39:56.359166594 +0000 UTC m=+0.398493489 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103525 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29197e6f542a5512d286d4d9f10acd SystemUUID:ec29197e-6f54-2a55-12d2-86d4d9f10acd BootID:36f920c6-b7f2-4196-884c-f2bfc0393d4c Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:da:e9:8e:18:95 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:da:e9:8e:18:95 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:d7:42:f6:3d:f6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 17:39:56.360820 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.360791 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 17:39:56.361068 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.360930 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 17:39:56.363270 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.363232 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 17:39:56.363434 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.363273 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-170.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 17:39:56.364146 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.364132 2577 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 17:39:56.364186 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.364148 2577 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 17:39:56.364186 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.364156 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 17:39:56.364186 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.364170 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 17:39:56.365969 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.365946 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 17:39:56.366080 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.366067 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 17:39:56.370857 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.370835 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 07 17:39:56.371003 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.370993 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 17:39:56.371069 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.371032 2577 kubelet.go:486] "Attempting to sync node with API server" Jul 07 17:39:56.371069 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.371067 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 17:39:56.371135 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.371092 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 17:39:56.371135 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.371108 2577 kubelet.go:398] "Adding apiserver pod source" Jul 07 17:39:56.371135 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.371119 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 17:39:56.371652 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.371636 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 17:39:56.371690 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.371666 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 17:39:56.374283 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.374261 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 17:39:56.376660 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.376640 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 17:39:56.376753 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.376672 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 17:39:56.378943 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.378927 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 17:39:56.379030 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.378950 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 17:39:56.379030 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.378960 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 17:39:56.379030 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.378968 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 17:39:56.379030 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.378978 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 17:39:56.379030 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.378984 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 17:39:56.379030 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.378993 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 17:39:56.379030 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.378998 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 17:39:56.379030 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.379005 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 17:39:56.379030 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.379013 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 17:39:56.379030 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.379027 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 17:39:56.379030 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.379033 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 17:39:56.380848 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.380829 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 17:39:56.380848 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.380847 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 17:39:56.381308 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.381280 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 17:39:56.381370 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.381303 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-170.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 17:39:56.384361 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.384342 2577 server.go:1268] "Started kubelet" Jul 07 17:39:56.384507 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.384451 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 17:39:56.384553 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.384539 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 17:39:56.384693 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.384432 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 17:39:56.388102 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.388066 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 17:39:56.388526 ip-10-0-129-170 systemd[1]: Started Kubernetes Kubelet. Jul 07 17:39:56.390605 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.390584 2577 server.go:310] "Adding debug handlers to kubelet server" Jul 07 17:39:56.393145 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.393118 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 17:39:56.394804 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.394781 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-170.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 17:39:56.396118 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.394891 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-170.ec2.internal.18c012af4294296d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-170.ec2.internal,UID:ip-10-0-129-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-170.ec2.internal,},FirstTimestamp:2026-07-07 17:39:56.384299373 +0000 UTC m=+0.423626266,LastTimestamp:2026-07-07 17:39:56.384299373 +0000 UTC m=+0.423626266,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-170.ec2.internal,}" Jul 07 17:39:56.400519 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.400435 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 17:39:56.400519 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.400493 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 17:39:56.400787 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.400645 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:56.401381 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.401345 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 17:39:56.401725 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.401613 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 17:39:56.401725 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.401631 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 17:39:56.401867 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.401738 2577 reconstruct.go:97] "Volume reconstruction finished" Jul 07 17:39:56.401867 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.401747 2577 reconciler.go:29] "Reconciler: start to sync state" Jul 07 17:39:56.402424 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.402405 2577 factory.go:153] Registering CRI-O factory Jul 07 17:39:56.402424 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.402425 2577 factory.go:223] Registration of the crio container factory successfully Jul 07 17:39:56.402531 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.402471 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 17:39:56.402531 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.402478 2577 factory.go:55] Registering systemd factory Jul 07 17:39:56.402531 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.402484 2577 factory.go:223] Registration of the systemd container factory successfully Jul 07 17:39:56.402531 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.402502 2577 factory.go:103] Registering Raw factory Jul 07 17:39:56.402531 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.402512 2577 manager.go:1196] Started watching for new ooms in manager Jul 07 17:39:56.403140 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.403123 2577 manager.go:349] Starting recovery of all containers Jul 07 17:39:56.405885 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.405823 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-170.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 17:39:56.406029 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.406005 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 17:39:56.413368 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.413163 2577 manager.go:354] Recovery completed Jul 07 17:39:56.418874 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.418850 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:39:56.420953 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.420933 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:39:56.421022 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.420972 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:39:56.421022 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.420983 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:39:56.421688 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.421669 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 17:39:56.421783 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.421697 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 17:39:56.421783 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.421676 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:39:56.421783 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.421719 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 07 17:39:56.422734 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.422716 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:39:56.422808 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.422745 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:39:56.422808 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.422757 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:39:56.423489 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.423377 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-170.ec2.internal.18c012af44c38120 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-170.ec2.internal,UID:ip-10-0-129-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-170.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-170.ec2.internal,},FirstTimestamp:2026-07-07 17:39:56.420956448 +0000 UTC m=+0.460283345,LastTimestamp:2026-07-07 17:39:56.420956448 +0000 UTC m=+0.460283345,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-170.ec2.internal,}" Jul 07 17:39:56.424117 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.424098 2577 policy_none.go:49] "None policy: Start" Jul 07 17:39:56.424201 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.424121 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 17:39:56.424201 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.424136 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 17:39:56.426189 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.426172 2577 policy_none.go:47] "Start" Jul 07 17:39:56.434710 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.434624 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-170.ec2.internal.18c012af44c3d4a4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-170.ec2.internal,UID:ip-10-0-129-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-170.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-170.ec2.internal,},FirstTimestamp:2026-07-07 17:39:56.420977828 +0000 UTC m=+0.460304722,LastTimestamp:2026-07-07 17:39:56.420977828 +0000 UTC m=+0.460304722,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-170.ec2.internal,}" Jul 07 17:39:56.438216 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.438138 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-170.ec2.internal.18c012af44c3fb9f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-170.ec2.internal,UID:ip-10-0-129-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-170.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-170.ec2.internal,},FirstTimestamp:2026-07-07 17:39:56.420987807 +0000 UTC m=+0.460314701,LastTimestamp:2026-07-07 17:39:56.420987807 +0000 UTC m=+0.460314701,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-170.ec2.internal,}" Jul 07 17:39:56.445386 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.445278 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-170.ec2.internal.18c012af44c38120\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-170.ec2.internal.18c012af44c38120 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-170.ec2.internal,UID:ip-10-0-129-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-170.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-170.ec2.internal,},FirstTimestamp:2026-07-07 17:39:56.420956448 +0000 UTC m=+0.460283345,LastTimestamp:2026-07-07 17:39:56.422732446 +0000 UTC m=+0.462059343,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-170.ec2.internal,}" Jul 07 17:39:56.447453 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.447430 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dcs4n" Jul 07 17:39:56.450793 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.450715 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-170.ec2.internal.18c012af44c3d4a4\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-170.ec2.internal.18c012af44c3d4a4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-170.ec2.internal,UID:ip-10-0-129-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-170.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-170.ec2.internal,},FirstTimestamp:2026-07-07 17:39:56.420977828 +0000 UTC m=+0.460304722,LastTimestamp:2026-07-07 17:39:56.422751873 +0000 UTC m=+0.462078773,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-170.ec2.internal,}" Jul 07 17:39:56.457319 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.457229 2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-170.ec2.internal.18c012af44c3fb9f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-170.ec2.internal.18c012af44c3fb9f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-170.ec2.internal,UID:ip-10-0-129-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-170.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-170.ec2.internal,},FirstTimestamp:2026-07-07 17:39:56.420987807 +0000 UTC m=+0.460314701,LastTimestamp:2026-07-07 17:39:56.422760787 +0000 UTC m=+0.462087684,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-170.ec2.internal,}" Jul 07 17:39:56.460843 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.460818 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dcs4n" Jul 07 17:39:56.464876 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.464856 2577 manager.go:337] "Starting Device Plugin manager" Jul 07 17:39:56.486223 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.464899 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 17:39:56.486223 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.464912 2577 server.go:87] "Starting device plugin registration server" Jul 07 17:39:56.486223 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.465234 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 17:39:56.486223 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.465249 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 17:39:56.486223 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.465377 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 17:39:56.486223 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.465464 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 17:39:56.486223 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.465471 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 17:39:56.486223 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.466003 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 17:39:56.486223 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.466079 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:56.486223 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.469845 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-170.ec2.internal.18c012af4788acdd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-170.ec2.internal,UID:ip-10-0-129-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-129-170.ec2.internal,},FirstTimestamp:2026-07-07 17:39:56.467432669 +0000 UTC m=+0.506759553,LastTimestamp:2026-07-07 17:39:56.467432669 +0000 UTC m=+0.506759553,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-170.ec2.internal,}" Jul 07 17:39:56.508295 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.508254 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 17:39:56.509701 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.509675 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 17:39:56.509790 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.509733 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 17:39:56.509790 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.509767 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 17:39:56.509876 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.509812 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 17:39:56.523583 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.523546 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:39:56.566227 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.566120 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:39:56.567191 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.567171 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:39:56.567318 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.567205 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:39:56.567318 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.567216 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:39:56.567318 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.567243 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.581651 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.581619 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.581651 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.581654 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-170.ec2.internal\": node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:56.610272 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.610231 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-170.ec2.internal"] Jul 07 17:39:56.610473 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.610320 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:39:56.611397 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.611374 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:39:56.611526 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.611413 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:39:56.611526 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.611427 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:39:56.613880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.613859 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:39:56.614001 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.613981 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.614083 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.614016 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:39:56.614797 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.614776 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:39:56.614879 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.614802 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:39:56.614879 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.614826 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:39:56.614879 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.614838 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:39:56.614980 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.614827 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:39:56.615018 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.614993 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:39:56.617208 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.617183 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.617300 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.617230 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:39:56.618027 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.618010 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:39:56.618143 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.618059 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:39:56.618143 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.618075 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:39:56.632086 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.632060 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:56.647085 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.647060 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-170.ec2.internal\" not found" node="ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.651698 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.651675 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-170.ec2.internal\" not found" node="ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.702862 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.702817 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7c76a1f86a523bd2550ec024c98736a4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal\" (UID: \"7c76a1f86a523bd2550ec024c98736a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.702862 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.702864 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c76a1f86a523bd2550ec024c98736a4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal\" (UID: \"7c76a1f86a523bd2550ec024c98736a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.703023 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.702916 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0973ca2b89d42fa29ef5474b01a07c7d-config\") pod \"kube-apiserver-proxy-ip-10-0-129-170.ec2.internal\" (UID: \"0973ca2b89d42fa29ef5474b01a07c7d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.733146 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.733115 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:56.803992 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.803956 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7c76a1f86a523bd2550ec024c98736a4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal\" (UID: \"7c76a1f86a523bd2550ec024c98736a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.804144 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.804002 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c76a1f86a523bd2550ec024c98736a4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal\" (UID: \"7c76a1f86a523bd2550ec024c98736a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.804144 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.804028 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0973ca2b89d42fa29ef5474b01a07c7d-config\") pod \"kube-apiserver-proxy-ip-10-0-129-170.ec2.internal\" (UID: \"0973ca2b89d42fa29ef5474b01a07c7d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.804144 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.804083 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7c76a1f86a523bd2550ec024c98736a4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal\" (UID: \"7c76a1f86a523bd2550ec024c98736a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.804144 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.804102 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0973ca2b89d42fa29ef5474b01a07c7d-config\") pod \"kube-apiserver-proxy-ip-10-0-129-170.ec2.internal\" (UID: \"0973ca2b89d42fa29ef5474b01a07c7d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.804144 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.804091 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c76a1f86a523bd2550ec024c98736a4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal\" (UID: \"7c76a1f86a523bd2550ec024c98736a4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.834307 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.834240 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:56.934958 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:56.934912 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:56.950117 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.950085 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal" Jul 07 17:39:56.955353 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:56.955334 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-170.ec2.internal" Jul 07 17:39:57.035424 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:57.035361 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:57.135928 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:57.135861 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:57.236536 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:57.236483 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:57.322950 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.322913 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 17:39:57.323578 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:57.323081 2577 request.go:1196] "Unexpected error when reading response body" err="read tcp 10.0.129.170:60512->3.229.59.57:6443: use of closed network connection" Jul 07 17:39:57.323578 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:57.323154 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: unexpected error when reading response body. Please retry. Original error: read tcp 10.0.129.170:60512->3.229.59.57:6443: use of closed network connection" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 17:39:57.323578 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.323087 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 17:39:57.337256 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:57.337215 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:57.438003 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:57.437909 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:57.468317 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.468276 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 17:34:57 +0000 UTC" deadline="2028-04-19 14:23:29.540845436 +0000 UTC" Jul 07 17:39:57.468317 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.468315 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15644h43m32.072533846s" Jul 07 17:39:57.538815 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:57.538771 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:57.598123 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.598056 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 17:39:57.599845 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.599820 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientMemory" Jul 07 17:39:57.599980 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.599859 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 17:39:57.599980 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.599876 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeHasSufficientPID" Jul 07 17:39:57.639903 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:57.639851 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:57.695327 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:57.695245 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c76a1f86a523bd2550ec024c98736a4.slice/crio-883e3dcdce60ce1664728b0fa393bb54096d50b63a11dfe24a2816dfa478105f WatchSource:0}: Error finding container 883e3dcdce60ce1664728b0fa393bb54096d50b63a11dfe24a2816dfa478105f: Status 404 returned error can't find the container with id 883e3dcdce60ce1664728b0fa393bb54096d50b63a11dfe24a2816dfa478105f Jul 07 17:39:57.700040 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.700005 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 17:39:57.706309 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:57.706281 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0973ca2b89d42fa29ef5474b01a07c7d.slice/crio-b161422b650352b3cc97b92c227bd512fef05c084c377c8f64bc55daaa1a8097 WatchSource:0}: Error finding container b161422b650352b3cc97b92c227bd512fef05c084c377c8f64bc55daaa1a8097: Status 404 returned error can't find the container with id b161422b650352b3cc97b92c227bd512fef05c084c377c8f64bc55daaa1a8097 Jul 07 17:39:57.741117 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:57.741071 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:57.819372 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.819339 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:39:57.841921 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:57.841880 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-170.ec2.internal\" not found" Jul 07 17:39:57.868807 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.868782 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:39:57.901818 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.901769 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal" Jul 07 17:39:57.916269 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.916244 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 17:39:57.917738 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.917716 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-170.ec2.internal" Jul 07 17:39:57.928065 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:57.928033 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 17:39:58.372445 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.372388 2577 apiserver.go:52] "Watching apiserver" Jul 07 17:39:58.382112 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.382054 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 17:39:58.382466 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.382437 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal","openshift-network-diagnostics/network-check-target-qtftl","openshift-ovn-kubernetes/ovnkube-node-tcrdq","kube-system/konnectivity-agent-qrfcq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr","openshift-dns/node-resolver-nck8x","openshift-image-registry/node-ca-t5q9p","openshift-multus/multus-6vdhw","openshift-multus/multus-additional-cni-plugins-tw276","openshift-multus/network-metrics-daemon-jdmfc","openshift-network-operator/iptables-alerter-n4rvd","kube-system/global-pull-secret-syncer-q4psm","kube-system/kube-apiserver-proxy-ip-10-0-129-170.ec2.internal","openshift-cluster-node-tuning-operator/tuned-9rnhd"] Jul 07 17:39:58.385558 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.385532 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t5q9p" Jul 07 17:39:58.387850 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.387821 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n4rvd" Jul 07 17:39:58.390220 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.390193 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 17:39:58.390220 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.390219 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.390809 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.390789 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 17:39:58.390908 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.390794 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 17:39:58.390970 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.390935 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9976q\"" Jul 07 17:39:58.391649 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.391627 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:39:58.391762 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.391704 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 17:39:58.393242 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.393218 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qrfcq" Jul 07 17:39:58.394058 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.394032 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 17:39:58.394183 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.394165 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 17:39:58.394322 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.394186 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 17:39:58.394381 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.394340 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 17:39:58.394381 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.394357 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 17:39:58.394381 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.394363 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 17:39:58.394526 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.394472 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2gpfj\"" Jul 07 17:39:58.394835 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.394818 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 17:39:58.395726 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.395707 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.396394 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.396376 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rz448\"" Jul 07 17:39:58.396480 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.396435 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lgncq\"" Jul 07 17:39:58.398293 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.398273 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nck8x" Jul 07 17:39:58.400893 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.400873 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.401904 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.401885 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 17:39:58.403433 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.403415 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.406335 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.406307 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 17:39:58.406463 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.406414 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:39:58.406575 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.406554 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 17:39:58.406644 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.406576 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 17:39:58.406644 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:58.406578 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:39:58.406942 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.406928 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hpwnw\"" Jul 07 17:39:58.407180 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.407167 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 17:39:58.407272 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.407256 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 17:39:58.407336 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.407307 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 17:39:58.407475 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.407457 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 17:39:58.407989 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.407836 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 17:39:58.407989 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.407863 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 17:39:58.407989 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.407920 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 17:39:58.408992 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.408969 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 17:39:58.409246 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.409126 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 17:39:58.409824 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.409332 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:39:58.411528 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.409969 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pg8gn\"" Jul 07 17:39:58.411528 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.410090 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qc82t\"" Jul 07 17:39:58.411528 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.410284 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 17:39:58.411528 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.410363 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2gdjk\"" Jul 07 17:39:58.411528 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.411363 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:39:58.411528 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:58.411459 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:39:58.412412 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412383 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-systemd-units\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.412503 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412441 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-run-systemd\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.412503 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412475 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/557c3dc0-235b-4285-ad82-77d2fbbfa75f-system-cni-dir\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.412503 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412498 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4db38d2b-e752-4fdf-81ed-d134f7831397-host\") pod \"node-ca-t5q9p\" (UID: \"4db38d2b-e752-4fdf-81ed-d134f7831397\") " pod="openshift-image-registry/node-ca-t5q9p" Jul 07 17:39:58.412634 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412520 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9pq2\" (UniqueName: \"kubernetes.io/projected/6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8-kube-api-access-x9pq2\") pod \"iptables-alerter-n4rvd\" (UID: \"6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8\") " pod="openshift-network-operator/iptables-alerter-n4rvd" Jul 07 17:39:58.412634 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412541 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-kubernetes\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.412634 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412563 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-run-netns\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.412634 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412588 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-cni-bin\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.412828 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412642 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1d8d149-d9ff-4b3c-851f-ec9de53109b6-hosts-file\") pod \"node-resolver-nck8x\" (UID: \"d1d8d149-d9ff-4b3c-851f-ec9de53109b6\") " pod="openshift-dns/node-resolver-nck8x" Jul 07 17:39:58.412828 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412692 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-var-lib-kubelet\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.412828 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412737 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-etc-openvswitch\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.412828 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412797 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-run-ovn-kubernetes\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.412993 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412835 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-run\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.412993 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412868 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-tuned\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.412993 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412923 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-tmp\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.412993 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412953 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d4zz\" (UniqueName: \"kubernetes.io/projected/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-kube-api-access-7d4zz\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.412993 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.412983 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-ovnkube-config\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.413281 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413017 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/557c3dc0-235b-4285-ad82-77d2fbbfa75f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.413281 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413106 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9prd\" (UniqueName: \"kubernetes.io/projected/d1d8d149-d9ff-4b3c-851f-ec9de53109b6-kube-api-access-f9prd\") pod \"node-resolver-nck8x\" (UID: \"d1d8d149-d9ff-4b3c-851f-ec9de53109b6\") " pod="openshift-dns/node-resolver-nck8x" Jul 07 17:39:58.413281 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413141 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-modprobe-d\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.413281 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413178 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.413281 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413206 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-sysconfig\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.413281 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413225 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-sysctl-conf\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.413281 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413264 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-env-overrides\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.413607 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413295 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/557c3dc0-235b-4285-ad82-77d2fbbfa75f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.413607 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413321 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74hwr\" (UniqueName: \"kubernetes.io/projected/557c3dc0-235b-4285-ad82-77d2fbbfa75f-kube-api-access-74hwr\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.413607 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413351 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4db38d2b-e752-4fdf-81ed-d134f7831397-serviceca\") pod \"node-ca-t5q9p\" (UID: \"4db38d2b-e752-4fdf-81ed-d134f7831397\") " pod="openshift-image-registry/node-ca-t5q9p" Jul 07 17:39:58.413607 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413372 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9gdb\" (UniqueName: \"kubernetes.io/projected/4db38d2b-e752-4fdf-81ed-d134f7831397-kube-api-access-d9gdb\") pod \"node-ca-t5q9p\" (UID: \"4db38d2b-e752-4fdf-81ed-d134f7831397\") " pod="openshift-image-registry/node-ca-t5q9p" Jul 07 17:39:58.413607 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413393 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-ovnkube-script-lib\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.413607 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413427 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvv96\" (UniqueName: \"kubernetes.io/projected/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-kube-api-access-rvv96\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.413607 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413470 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/557c3dc0-235b-4285-ad82-77d2fbbfa75f-os-release\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.413607 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413505 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-host\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.413607 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413554 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-run-openvswitch\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.413607 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413584 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8-host-slash\") pod \"iptables-alerter-n4rvd\" (UID: \"6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8\") " pod="openshift-network-operator/iptables-alerter-n4rvd" Jul 07 17:39:58.413607 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413604 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-node-log\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413631 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8-iptables-alerter-script\") pod \"iptables-alerter-n4rvd\" (UID: \"6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8\") " pod="openshift-network-operator/iptables-alerter-n4rvd" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413662 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-sysctl-d\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413697 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-var-lib-openvswitch\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413749 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/557c3dc0-235b-4285-ad82-77d2fbbfa75f-cni-binary-copy\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413780 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/557c3dc0-235b-4285-ad82-77d2fbbfa75f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413815 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb7150e3-b3f8-4086-883c-2a2d4720246b-konnectivity-ca\") pod \"konnectivity-agent-qrfcq\" (UID: \"cb7150e3-b3f8-4086-883c-2a2d4720246b\") " pod="kube-system/konnectivity-agent-qrfcq" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413841 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413848 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-socket-dir\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413881 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-device-dir\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413911 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-etc-selinux\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413934 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-systemd\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.413980 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-kubelet\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.414005 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vmcm\" (UniqueName: \"kubernetes.io/projected/e7f9d035-66c6-4888-89af-15077571b05f-kube-api-access-8vmcm\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.414077 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-registration-dir\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.414125 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-sys-fs\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.414184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.414165 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-lib-modules\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.414990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.414211 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-run-ovn\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.414990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.414241 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.414990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.414303 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1d8d149-d9ff-4b3c-851f-ec9de53109b6-tmp-dir\") pod \"node-resolver-nck8x\" (UID: \"d1d8d149-d9ff-4b3c-851f-ec9de53109b6\") " pod="openshift-dns/node-resolver-nck8x" Jul 07 17:39:58.414990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.414328 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-log-socket\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.414990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.414359 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-cni-netd\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.414990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.414398 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb7150e3-b3f8-4086-883c-2a2d4720246b-agent-certs\") pod \"konnectivity-agent-qrfcq\" (UID: \"cb7150e3-b3f8-4086-883c-2a2d4720246b\") " pod="kube-system/konnectivity-agent-qrfcq" Jul 07 17:39:58.414990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.414431 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/557c3dc0-235b-4285-ad82-77d2fbbfa75f-cnibin\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.414990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.414450 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-slash\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.414990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.414500 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-ovn-node-metrics-cert\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.414990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.414530 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-sys\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.416769 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.416742 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:39:58.416913 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:58.416822 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:39:58.418360 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.418336 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4vllp\"" Jul 07 17:39:58.418632 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.418614 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 17:39:58.430957 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.430835 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 17:39:58.465838 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.465801 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zh5d2" Jul 07 17:39:58.477392 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.477355 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zh5d2" Jul 07 17:39:58.503276 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.503254 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 17:39:58.514726 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.514691 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.514726 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.514739 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-sysconfig\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.515002 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.514766 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-sysctl-conf\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.515002 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.514792 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-env-overrides\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.515002 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.514818 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/557c3dc0-235b-4285-ad82-77d2fbbfa75f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.515002 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.514838 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.515002 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.514847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74hwr\" (UniqueName: \"kubernetes.io/projected/557c3dc0-235b-4285-ad82-77d2fbbfa75f-kube-api-access-74hwr\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.515002 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.514863 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-sysconfig\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.515002 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.514872 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4db38d2b-e752-4fdf-81ed-d134f7831397-serviceca\") pod \"node-ca-t5q9p\" (UID: \"4db38d2b-e752-4fdf-81ed-d134f7831397\") " pod="openshift-image-registry/node-ca-t5q9p" Jul 07 17:39:58.515002 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.514911 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9gdb\" (UniqueName: \"kubernetes.io/projected/4db38d2b-e752-4fdf-81ed-d134f7831397-kube-api-access-d9gdb\") pod \"node-ca-t5q9p\" (UID: \"4db38d2b-e752-4fdf-81ed-d134f7831397\") " pod="openshift-image-registry/node-ca-t5q9p" Jul 07 17:39:58.515002 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.514934 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-ovnkube-script-lib\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.515002 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.514953 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvv96\" (UniqueName: \"kubernetes.io/projected/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-kube-api-access-rvv96\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.515002 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.514977 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-sysctl-conf\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.515002 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.514994 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/557c3dc0-235b-4285-ad82-77d2fbbfa75f-os-release\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515030 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-host\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515085 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/970a1d18-134b-417d-8b58-aa2ddd03d5bf-kubelet-config\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515117 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/557c3dc0-235b-4285-ad82-77d2fbbfa75f-os-release\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515146 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-run-openvswitch\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515172 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8-host-slash\") pod \"iptables-alerter-n4rvd\" (UID: \"6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8\") " pod="openshift-network-operator/iptables-alerter-n4rvd" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515197 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-node-log\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515226 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8-iptables-alerter-script\") pod \"iptables-alerter-n4rvd\" (UID: \"6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8\") " pod="openshift-network-operator/iptables-alerter-n4rvd" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515008 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-170.ec2.internal" event={"ID":"0973ca2b89d42fa29ef5474b01a07c7d","Type":"ContainerStarted","Data":"b161422b650352b3cc97b92c227bd512fef05c084c377c8f64bc55daaa1a8097"} Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515253 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-sysctl-d\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515279 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-system-cni-dir\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515305 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-hostroot\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515324 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-host\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515333 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-var-lib-openvswitch\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515371 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-var-lib-openvswitch\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515372 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/557c3dc0-235b-4285-ad82-77d2fbbfa75f-cni-binary-copy\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515443 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/557c3dc0-235b-4285-ad82-77d2fbbfa75f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.515684 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515448 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-env-overrides\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515473 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb7150e3-b3f8-4086-883c-2a2d4720246b-konnectivity-ca\") pod \"konnectivity-agent-qrfcq\" (UID: \"cb7150e3-b3f8-4086-883c-2a2d4720246b\") " pod="kube-system/konnectivity-agent-qrfcq" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515499 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-socket-dir\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515526 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-device-dir\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515558 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-sysctl-d\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515591 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-etc-selinux\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515604 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/557c3dc0-235b-4285-ad82-77d2fbbfa75f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515613 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/557c3dc0-235b-4285-ad82-77d2fbbfa75f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515619 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-systemd\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515645 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-kubelet\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515661 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-ovnkube-script-lib\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515693 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-run-openvswitch\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515694 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-systemd\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515722 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-device-dir\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515757 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-node-log\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515759 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-etc-selinux\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515795 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-socket-dir\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.516516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515814 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vmcm\" (UniqueName: \"kubernetes.io/projected/e7f9d035-66c6-4888-89af-15077571b05f-kube-api-access-8vmcm\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515848 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-cnibin\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515877 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-cni-binary-copy\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515929 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-run-netns\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515931 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/557c3dc0-235b-4285-ad82-77d2fbbfa75f-cni-binary-copy\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515970 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8-host-slash\") pod \"iptables-alerter-n4rvd\" (UID: \"6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8\") " pod="openshift-network-operator/iptables-alerter-n4rvd" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515955 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-var-lib-cni-bin\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.515998 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4db38d2b-e752-4fdf-81ed-d134f7831397-serviceca\") pod \"node-ca-t5q9p\" (UID: \"4db38d2b-e752-4fdf-81ed-d134f7831397\") " pod="openshift-image-registry/node-ca-t5q9p" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516033 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-kubelet\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516029 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-multus-conf-dir\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516086 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516140 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-registration-dir\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516171 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-sys-fs\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516221 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-lib-modules\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516227 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-registration-dir\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516271 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7f9d035-66c6-4888-89af-15077571b05f-sys-fs\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516284 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-run-ovn\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.517332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516319 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516321 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8-iptables-alerter-script\") pod \"iptables-alerter-n4rvd\" (UID: \"6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8\") " pod="openshift-network-operator/iptables-alerter-n4rvd" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516333 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb7150e3-b3f8-4086-883c-2a2d4720246b-konnectivity-ca\") pod \"konnectivity-agent-qrfcq\" (UID: \"cb7150e3-b3f8-4086-883c-2a2d4720246b\") " pod="kube-system/konnectivity-agent-qrfcq" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516354 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-lib-modules\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516366 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-multus-socket-dir-parent\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516376 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-run-ovn\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516386 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516424 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1d8d149-d9ff-4b3c-851f-ec9de53109b6-tmp-dir\") pod \"node-resolver-nck8x\" (UID: \"d1d8d149-d9ff-4b3c-851f-ec9de53109b6\") " pod="openshift-dns/node-resolver-nck8x" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516442 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-log-socket\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516467 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-cni-netd\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516496 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb7150e3-b3f8-4086-883c-2a2d4720246b-agent-certs\") pod \"konnectivity-agent-qrfcq\" (UID: \"cb7150e3-b3f8-4086-883c-2a2d4720246b\") " pod="kube-system/konnectivity-agent-qrfcq" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516569 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-run-k8s-cni-cncf-io\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516546 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-cni-netd\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516592 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-run-multus-certs\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516637 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/557c3dc0-235b-4285-ad82-77d2fbbfa75f-cnibin\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516664 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-log-socket\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516678 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-var-lib-cni-multus\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.518717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516713 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-var-lib-kubelet\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516719 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/557c3dc0-235b-4285-ad82-77d2fbbfa75f-cnibin\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516743 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-etc-kubernetes\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516745 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1d8d149-d9ff-4b3c-851f-ec9de53109b6-tmp-dir\") pod \"node-resolver-nck8x\" (UID: \"d1d8d149-d9ff-4b3c-851f-ec9de53109b6\") " pod="openshift-dns/node-resolver-nck8x" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516762 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-slash\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516780 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-ovn-node-metrics-cert\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516830 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-sys\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516834 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-slash\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516872 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-sys\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516890 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-systemd-units\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516901 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516915 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-run-systemd\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516954 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/557c3dc0-235b-4285-ad82-77d2fbbfa75f-system-cni-dir\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516973 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-systemd-units\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516995 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/557c3dc0-235b-4285-ad82-77d2fbbfa75f-system-cni-dir\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517005 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-run-systemd\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.516979 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4db38d2b-e752-4fdf-81ed-d134f7831397-host\") pod \"node-ca-t5q9p\" (UID: \"4db38d2b-e752-4fdf-81ed-d134f7831397\") " pod="openshift-image-registry/node-ca-t5q9p" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517026 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4db38d2b-e752-4fdf-81ed-d134f7831397-host\") pod \"node-ca-t5q9p\" (UID: \"4db38d2b-e752-4fdf-81ed-d134f7831397\") " pod="openshift-image-registry/node-ca-t5q9p" Jul 07 17:39:58.519595 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517072 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9pq2\" (UniqueName: \"kubernetes.io/projected/6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8-kube-api-access-x9pq2\") pod \"iptables-alerter-n4rvd\" (UID: \"6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8\") " pod="openshift-network-operator/iptables-alerter-n4rvd" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517097 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-kubernetes\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517114 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-os-release\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517132 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-run-netns\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517160 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-cni-bin\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517181 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-run-netns\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517188 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs\") pod \"network-metrics-daemon-jdmfc\" (UID: \"9541dd58-70fc-4069-bdba-f4726c87e4d2\") " pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517179 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-kubernetes\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517218 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8ptn\" (UniqueName: \"kubernetes.io/projected/9541dd58-70fc-4069-bdba-f4726c87e4d2-kube-api-access-n8ptn\") pod \"network-metrics-daemon-jdmfc\" (UID: \"9541dd58-70fc-4069-bdba-f4726c87e4d2\") " pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517257 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-cni-bin\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517420 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1d8d149-d9ff-4b3c-851f-ec9de53109b6-hosts-file\") pod \"node-resolver-nck8x\" (UID: \"d1d8d149-d9ff-4b3c-851f-ec9de53109b6\") " pod="openshift-dns/node-resolver-nck8x" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517448 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-var-lib-kubelet\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517480 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-multus-daemon-config\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517505 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7db2z\" (UniqueName: \"kubernetes.io/projected/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-kube-api-access-7db2z\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517535 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cw84\" (UniqueName: \"kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84\") pod \"network-check-target-qtftl\" (UID: \"4f7c694f-4fcf-4310-9772-12030534d40a\") " pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517548 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1d8d149-d9ff-4b3c-851f-ec9de53109b6-hosts-file\") pod \"node-resolver-nck8x\" (UID: \"d1d8d149-d9ff-4b3c-851f-ec9de53109b6\") " pod="openshift-dns/node-resolver-nck8x" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517564 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-etc-openvswitch\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.520510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517590 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-var-lib-kubelet\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-run-ovn-kubernetes\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517622 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-etc-openvswitch\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517627 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-run\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517638 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-host-run-ovn-kubernetes\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517659 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-tuned\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517677 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-run\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517686 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-tmp\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517715 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d4zz\" (UniqueName: \"kubernetes.io/projected/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-kube-api-access-7d4zz\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517742 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-multus-cni-dir\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517768 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-ovnkube-config\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517797 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/557c3dc0-235b-4285-ad82-77d2fbbfa75f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517824 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9prd\" (UniqueName: \"kubernetes.io/projected/d1d8d149-d9ff-4b3c-851f-ec9de53109b6-kube-api-access-f9prd\") pod \"node-resolver-nck8x\" (UID: \"d1d8d149-d9ff-4b3c-851f-ec9de53109b6\") " pod="openshift-dns/node-resolver-nck8x" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517856 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-modprobe-d\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.517879 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/970a1d18-134b-417d-8b58-aa2ddd03d5bf-dbus\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.518461 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/557c3dc0-235b-4285-ad82-77d2fbbfa75f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.518490 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-ovnkube-config\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.521287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.518509 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-modprobe-d\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.522109 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.519206 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal" event={"ID":"7c76a1f86a523bd2550ec024c98736a4","Type":"ContainerStarted","Data":"883e3dcdce60ce1664728b0fa393bb54096d50b63a11dfe24a2816dfa478105f"} Jul 07 17:39:58.522109 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.521160 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-etc-tuned\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.522109 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.521421 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-ovn-node-metrics-cert\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.522109 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.521529 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-tmp\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.522109 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.521797 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb7150e3-b3f8-4086-883c-2a2d4720246b-agent-certs\") pod \"konnectivity-agent-qrfcq\" (UID: \"cb7150e3-b3f8-4086-883c-2a2d4720246b\") " pod="kube-system/konnectivity-agent-qrfcq" Jul 07 17:39:58.550871 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.550822 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vmcm\" (UniqueName: \"kubernetes.io/projected/e7f9d035-66c6-4888-89af-15077571b05f-kube-api-access-8vmcm\") pod \"aws-ebs-csi-driver-node-nnwjr\" (UID: \"e7f9d035-66c6-4888-89af-15077571b05f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.551080 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.550870 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9gdb\" (UniqueName: \"kubernetes.io/projected/4db38d2b-e752-4fdf-81ed-d134f7831397-kube-api-access-d9gdb\") pod \"node-ca-t5q9p\" (UID: \"4db38d2b-e752-4fdf-81ed-d134f7831397\") " pod="openshift-image-registry/node-ca-t5q9p" Jul 07 17:39:58.551080 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.550825 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9prd\" (UniqueName: \"kubernetes.io/projected/d1d8d149-d9ff-4b3c-851f-ec9de53109b6-kube-api-access-f9prd\") pod \"node-resolver-nck8x\" (UID: \"d1d8d149-d9ff-4b3c-851f-ec9de53109b6\") " pod="openshift-dns/node-resolver-nck8x" Jul 07 17:39:58.551491 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.551459 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9pq2\" (UniqueName: \"kubernetes.io/projected/6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8-kube-api-access-x9pq2\") pod \"iptables-alerter-n4rvd\" (UID: \"6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8\") " pod="openshift-network-operator/iptables-alerter-n4rvd" Jul 07 17:39:58.551929 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.551904 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d4zz\" (UniqueName: \"kubernetes.io/projected/76382e09-f5ec-4ea9-8bfc-42a4a31c2638-kube-api-access-7d4zz\") pod \"tuned-9rnhd\" (UID: \"76382e09-f5ec-4ea9-8bfc-42a4a31c2638\") " pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.552779 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.552749 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74hwr\" (UniqueName: \"kubernetes.io/projected/557c3dc0-235b-4285-ad82-77d2fbbfa75f-kube-api-access-74hwr\") pod \"multus-additional-cni-plugins-tw276\" (UID: \"557c3dc0-235b-4285-ad82-77d2fbbfa75f\") " pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.553686 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.553663 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvv96\" (UniqueName: \"kubernetes.io/projected/2fcb0730-da1f-4113-aa11-4232d0c7dbf8-kube-api-access-rvv96\") pod \"ovnkube-node-tcrdq\" (UID: \"2fcb0730-da1f-4113-aa11-4232d0c7dbf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.618635 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.618593 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/970a1d18-134b-417d-8b58-aa2ddd03d5bf-kubelet-config\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:39:58.618635 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.618647 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-system-cni-dir\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.618909 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.618674 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-hostroot\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.618909 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.618709 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-cnibin\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.618909 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.618734 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-cni-binary-copy\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.618909 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.618761 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-run-netns\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.618909 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.618786 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-var-lib-cni-bin\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.618909 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.618812 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-multus-conf-dir\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.618909 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.618837 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:39:58.618909 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.618868 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-multus-socket-dir-parent\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.618909 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.618897 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-run-k8s-cni-cncf-io\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.619411 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.618924 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-run-multus-certs\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.619411 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.618955 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-var-lib-cni-multus\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.619411 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.618981 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-var-lib-kubelet\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.619411 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.619007 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-etc-kubernetes\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.619411 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.619039 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-os-release\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.619411 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.619067 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs\") pod \"network-metrics-daemon-jdmfc\" (UID: \"9541dd58-70fc-4069-bdba-f4726c87e4d2\") " pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:39:58.619411 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.619114 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8ptn\" (UniqueName: \"kubernetes.io/projected/9541dd58-70fc-4069-bdba-f4726c87e4d2-kube-api-access-n8ptn\") pod \"network-metrics-daemon-jdmfc\" (UID: \"9541dd58-70fc-4069-bdba-f4726c87e4d2\") " pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:39:58.619411 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.619144 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-multus-daemon-config\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.619411 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.619182 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7db2z\" (UniqueName: \"kubernetes.io/projected/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-kube-api-access-7db2z\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.619411 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.619208 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cw84\" (UniqueName: \"kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84\") pod \"network-check-target-qtftl\" (UID: \"4f7c694f-4fcf-4310-9772-12030534d40a\") " pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:39:58.619411 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.619241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-multus-cni-dir\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.619411 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.619270 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/970a1d18-134b-417d-8b58-aa2ddd03d5bf-dbus\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:39:58.620045 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.619489 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/970a1d18-134b-417d-8b58-aa2ddd03d5bf-dbus\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:39:58.620045 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.619554 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/970a1d18-134b-417d-8b58-aa2ddd03d5bf-kubelet-config\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:39:58.620045 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.619605 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-system-cni-dir\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.620045 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.619642 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-hostroot\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.620045 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.619691 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-cnibin\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.620397 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.620352 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-cni-binary-copy\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.620454 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.620438 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-run-netns\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.620510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.620492 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-var-lib-cni-bin\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.620567 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.620541 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-multus-conf-dir\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.620667 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:58.620649 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:39:58.620748 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:58.620736 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret podName:970a1d18-134b-417d-8b58-aa2ddd03d5bf nodeName:}" failed. No retries permitted until 2026-07-07 17:39:59.120704949 +0000 UTC m=+3.160031848 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret") pod "global-pull-secret-syncer-q4psm" (UID: "970a1d18-134b-417d-8b58-aa2ddd03d5bf") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:39:58.620843 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.620826 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-multus-socket-dir-parent\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.620911 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.620877 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-run-k8s-cni-cncf-io\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.620967 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.620929 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-run-multus-certs\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.621022 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.620988 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-var-lib-cni-multus\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.621074 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.621031 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-host-var-lib-kubelet\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.621142 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.621082 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-etc-kubernetes\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.621202 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.621174 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-os-release\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.621280 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:58.621264 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:39:58.621343 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:58.621331 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs podName:9541dd58-70fc-4069-bdba-f4726c87e4d2 nodeName:}" failed. No retries permitted until 2026-07-07 17:39:59.121313763 +0000 UTC m=+3.160640649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs") pod "network-metrics-daemon-jdmfc" (UID: "9541dd58-70fc-4069-bdba-f4726c87e4d2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:39:58.622048 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.622028 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-multus-daemon-config\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.622514 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.622444 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-multus-cni-dir\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.638163 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:58.638106 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:39:58.638163 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:58.638166 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:39:58.638419 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:58.638185 2577 projected.go:196] Error preparing data for projected volume kube-api-access-4cw84 for pod openshift-network-diagnostics/network-check-target-qtftl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:39:58.638419 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:58.638266 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84 podName:4f7c694f-4fcf-4310-9772-12030534d40a nodeName:}" failed. No retries permitted until 2026-07-07 17:39:59.138240329 +0000 UTC m=+3.177567230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4cw84" (UniqueName: "kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84") pod "network-check-target-qtftl" (UID: "4f7c694f-4fcf-4310-9772-12030534d40a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:39:58.640330 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.640294 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7db2z\" (UniqueName: \"kubernetes.io/projected/7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71-kube-api-access-7db2z\") pod \"multus-6vdhw\" (UID: \"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71\") " pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.641502 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.641447 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8ptn\" (UniqueName: \"kubernetes.io/projected/9541dd58-70fc-4069-bdba-f4726c87e4d2-kube-api-access-n8ptn\") pod \"network-metrics-daemon-jdmfc\" (UID: \"9541dd58-70fc-4069-bdba-f4726c87e4d2\") " pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:39:58.696387 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.696340 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t5q9p" Jul 07 17:39:58.704203 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.704167 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n4rvd" Jul 07 17:39:58.707929 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:58.707876 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4db38d2b_e752_4fdf_81ed_d134f7831397.slice/crio-d80f3c4cba4b0dbbc6420930902561d2e0b7dba984c64b28cb0d4e24e7bc4fc7 WatchSource:0}: Error finding container d80f3c4cba4b0dbbc6420930902561d2e0b7dba984c64b28cb0d4e24e7bc4fc7: Status 404 returned error can't find the container with id d80f3c4cba4b0dbbc6420930902561d2e0b7dba984c64b28cb0d4e24e7bc4fc7 Jul 07 17:39:58.714172 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.714147 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:39:58.717832 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:58.717794 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1fec5d_b6fb_44ac_bd0d_341cfd7c56c8.slice/crio-5c7655130f8b3de7debaf63ff7681faa41550cc2d561e4c8ba0c8f9263f3a003 WatchSource:0}: Error finding container 5c7655130f8b3de7debaf63ff7681faa41550cc2d561e4c8ba0c8f9263f3a003: Status 404 returned error can't find the container with id 5c7655130f8b3de7debaf63ff7681faa41550cc2d561e4c8ba0c8f9263f3a003 Jul 07 17:39:58.721580 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.721359 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qrfcq" Jul 07 17:39:58.726986 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:58.726947 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fcb0730_da1f_4113_aa11_4232d0c7dbf8.slice/crio-c532a5996264a766a845eeff8926964c00e05f7425825a6e095b2f77a3f3c503 WatchSource:0}: Error finding container c532a5996264a766a845eeff8926964c00e05f7425825a6e095b2f77a3f3c503: Status 404 returned error can't find the container with id c532a5996264a766a845eeff8926964c00e05f7425825a6e095b2f77a3f3c503 Jul 07 17:39:58.728088 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.728037 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" Jul 07 17:39:58.734167 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.734132 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nck8x" Jul 07 17:39:58.734739 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:58.734707 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb7150e3_b3f8_4086_883c_2a2d4720246b.slice/crio-88614b79d8e7adc3a224c39dd836f00ccb48b93288fb1d1a26865b2d5c0f01d4 WatchSource:0}: Error finding container 88614b79d8e7adc3a224c39dd836f00ccb48b93288fb1d1a26865b2d5c0f01d4: Status 404 returned error can't find the container with id 88614b79d8e7adc3a224c39dd836f00ccb48b93288fb1d1a26865b2d5c0f01d4 Jul 07 17:39:58.740572 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.740546 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tw276" Jul 07 17:39:58.741689 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:58.741640 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7f9d035_66c6_4888_89af_15077571b05f.slice/crio-75b025859cc520e4c02d557e8d41feac21a5b4bff4f8a8add56d9ffd1f328c10 WatchSource:0}: Error finding container 75b025859cc520e4c02d557e8d41feac21a5b4bff4f8a8add56d9ffd1f328c10: Status 404 returned error can't find the container with id 75b025859cc520e4c02d557e8d41feac21a5b4bff4f8a8add56d9ffd1f328c10 Jul 07 17:39:58.747564 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.747384 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" Jul 07 17:39:58.749278 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:58.749245 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d8d149_d9ff_4b3c_851f_ec9de53109b6.slice/crio-e56d47a2685217121a585f436a4a7548935e8d2f49d6a202e870c06c37d0c980 WatchSource:0}: Error finding container e56d47a2685217121a585f436a4a7548935e8d2f49d6a202e870c06c37d0c980: Status 404 returned error can't find the container with id e56d47a2685217121a585f436a4a7548935e8d2f49d6a202e870c06c37d0c980 Jul 07 17:39:58.751636 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.751605 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6vdhw" Jul 07 17:39:58.755395 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:58.755348 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod557c3dc0_235b_4285_ad82_77d2fbbfa75f.slice/crio-9da5b9668f16b100108f2f95017713df321b8f7d7bcd3267c22604fab4b547a0 WatchSource:0}: Error finding container 9da5b9668f16b100108f2f95017713df321b8f7d7bcd3267c22604fab4b547a0: Status 404 returned error can't find the container with id 9da5b9668f16b100108f2f95017713df321b8f7d7bcd3267c22604fab4b547a0 Jul 07 17:39:58.766461 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:58.766425 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76382e09_f5ec_4ea9_8bfc_42a4a31c2638.slice/crio-3f5c8a06dcf00cd6363455df637312684946d920ddd4f0d9bb30de2c2bb59ed8 WatchSource:0}: Error finding container 3f5c8a06dcf00cd6363455df637312684946d920ddd4f0d9bb30de2c2bb59ed8: Status 404 returned error can't find the container with id 3f5c8a06dcf00cd6363455df637312684946d920ddd4f0d9bb30de2c2bb59ed8 Jul 07 17:39:58.769067 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:58.769043 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:39:58.769681 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:39:58.769651 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b0b588c_d0f2_4e5f_9c0d_11e7e5a0fb71.slice/crio-8e2e14aa815105cacc24c5683593500883ec3d578cf20b4a77260951119bcf5d WatchSource:0}: Error finding container 8e2e14aa815105cacc24c5683593500883ec3d578cf20b4a77260951119bcf5d: Status 404 returned error can't find the container with id 8e2e14aa815105cacc24c5683593500883ec3d578cf20b4a77260951119bcf5d Jul 07 17:39:59.123318 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.123277 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs\") pod \"network-metrics-daemon-jdmfc\" (UID: \"9541dd58-70fc-4069-bdba-f4726c87e4d2\") " pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:39:59.123505 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.123382 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:39:59.123505 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:59.123486 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:39:59.123618 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:59.123537 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:39:59.123618 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:59.123572 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs podName:9541dd58-70fc-4069-bdba-f4726c87e4d2 nodeName:}" failed. No retries permitted until 2026-07-07 17:40:00.123550377 +0000 UTC m=+4.162877275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs") pod "network-metrics-daemon-jdmfc" (UID: "9541dd58-70fc-4069-bdba-f4726c87e4d2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:39:59.123618 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:59.123590 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret podName:970a1d18-134b-417d-8b58-aa2ddd03d5bf nodeName:}" failed. No retries permitted until 2026-07-07 17:40:00.123582699 +0000 UTC m=+4.162909582 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret") pod "global-pull-secret-syncer-q4psm" (UID: "970a1d18-134b-417d-8b58-aa2ddd03d5bf") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:39:59.223888 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.223775 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cw84\" (UniqueName: \"kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84\") pod \"network-check-target-qtftl\" (UID: \"4f7c694f-4fcf-4310-9772-12030534d40a\") " pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:39:59.224070 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:59.223964 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:39:59.224070 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:59.223992 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:39:59.224070 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:59.224003 2577 projected.go:196] Error preparing data for projected volume kube-api-access-4cw84 for pod openshift-network-diagnostics/network-check-target-qtftl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:39:59.224199 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:59.224073 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84 podName:4f7c694f-4fcf-4310-9772-12030534d40a nodeName:}" failed. No retries permitted until 2026-07-07 17:40:00.224045049 +0000 UTC m=+4.263371931 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4cw84" (UniqueName: "kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84") pod "network-check-target-qtftl" (UID: "4f7c694f-4fcf-4310-9772-12030534d40a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:39:59.481450 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.481291 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 17:34:59 +0000 UTC" deadline="2027-12-13 14:01:31.8181896 +0000 UTC" Jul 07 17:39:59.481450 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.481336 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12572h21m32.336858959s" Jul 07 17:39:59.511442 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.510861 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:39:59.511442 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:39:59.510999 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:39:59.524690 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.524639 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" event={"ID":"76382e09-f5ec-4ea9-8bfc-42a4a31c2638","Type":"ContainerStarted","Data":"3f5c8a06dcf00cd6363455df637312684946d920ddd4f0d9bb30de2c2bb59ed8"} Jul 07 17:39:59.530417 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.530380 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw276" event={"ID":"557c3dc0-235b-4285-ad82-77d2fbbfa75f","Type":"ContainerStarted","Data":"9da5b9668f16b100108f2f95017713df321b8f7d7bcd3267c22604fab4b547a0"} Jul 07 17:39:59.534226 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.534072 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" event={"ID":"e7f9d035-66c6-4888-89af-15077571b05f","Type":"ContainerStarted","Data":"75b025859cc520e4c02d557e8d41feac21a5b4bff4f8a8add56d9ffd1f328c10"} Jul 07 17:39:59.538573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.538527 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qrfcq" event={"ID":"cb7150e3-b3f8-4086-883c-2a2d4720246b","Type":"ContainerStarted","Data":"88614b79d8e7adc3a224c39dd836f00ccb48b93288fb1d1a26865b2d5c0f01d4"} Jul 07 17:39:59.540654 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.540611 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" event={"ID":"2fcb0730-da1f-4113-aa11-4232d0c7dbf8","Type":"ContainerStarted","Data":"c532a5996264a766a845eeff8926964c00e05f7425825a6e095b2f77a3f3c503"} Jul 07 17:39:59.542242 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.542174 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t5q9p" event={"ID":"4db38d2b-e752-4fdf-81ed-d134f7831397","Type":"ContainerStarted","Data":"d80f3c4cba4b0dbbc6420930902561d2e0b7dba984c64b28cb0d4e24e7bc4fc7"} Jul 07 17:39:59.544083 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.543981 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6vdhw" event={"ID":"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71","Type":"ContainerStarted","Data":"8e2e14aa815105cacc24c5683593500883ec3d578cf20b4a77260951119bcf5d"} Jul 07 17:39:59.545347 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.545248 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nck8x" event={"ID":"d1d8d149-d9ff-4b3c-851f-ec9de53109b6","Type":"ContainerStarted","Data":"e56d47a2685217121a585f436a4a7548935e8d2f49d6a202e870c06c37d0c980"} Jul 07 17:39:59.547540 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.547471 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n4rvd" event={"ID":"6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8","Type":"ContainerStarted","Data":"5c7655130f8b3de7debaf63ff7681faa41550cc2d561e4c8ba0c8f9263f3a003"} Jul 07 17:39:59.665879 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:39:59.665839 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 17:40:00.131750 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:00.131699 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:00.131982 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:00.131779 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs\") pod \"network-metrics-daemon-jdmfc\" (UID: \"9541dd58-70fc-4069-bdba-f4726c87e4d2\") " pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:00.131982 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:00.131953 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:40:00.132141 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:00.132019 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs podName:9541dd58-70fc-4069-bdba-f4726c87e4d2 nodeName:}" failed. No retries permitted until 2026-07-07 17:40:02.131999708 +0000 UTC m=+6.171326593 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs") pod "network-metrics-daemon-jdmfc" (UID: "9541dd58-70fc-4069-bdba-f4726c87e4d2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:40:00.132540 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:00.132512 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:40:00.132648 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:00.132579 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret podName:970a1d18-134b-417d-8b58-aa2ddd03d5bf nodeName:}" failed. No retries permitted until 2026-07-07 17:40:02.132562113 +0000 UTC m=+6.171889003 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret") pod "global-pull-secret-syncer-q4psm" (UID: "970a1d18-134b-417d-8b58-aa2ddd03d5bf") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:40:00.232820 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:00.232707 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cw84\" (UniqueName: \"kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84\") pod \"network-check-target-qtftl\" (UID: \"4f7c694f-4fcf-4310-9772-12030534d40a\") " pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:00.233155 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:00.232929 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:40:00.233155 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:00.232958 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:40:00.233155 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:00.232972 2577 projected.go:196] Error preparing data for projected volume kube-api-access-4cw84 for pod openshift-network-diagnostics/network-check-target-qtftl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:40:00.233155 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:00.233038 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84 podName:4f7c694f-4fcf-4310-9772-12030534d40a nodeName:}" failed. No retries permitted until 2026-07-07 17:40:02.233017065 +0000 UTC m=+6.272343963 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4cw84" (UniqueName: "kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84") pod "network-check-target-qtftl" (UID: "4f7c694f-4fcf-4310-9772-12030534d40a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:40:00.482463 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:00.482342 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 17:34:59 +0000 UTC" deadline="2027-12-13 14:59:07.167409127 +0000 UTC" Jul 07 17:40:00.482463 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:00.482415 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12573h19m6.684998474s" Jul 07 17:40:00.512585 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:00.512289 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:00.512812 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:00.512711 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:00.513335 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:00.513311 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:00.513460 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:00.513436 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:01.510469 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:01.510267 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:01.510469 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:01.510429 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:02.149388 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:02.148391 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:02.149388 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:02.148467 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs\") pod \"network-metrics-daemon-jdmfc\" (UID: \"9541dd58-70fc-4069-bdba-f4726c87e4d2\") " pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:02.149388 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:02.148622 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:40:02.149388 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:02.148692 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs podName:9541dd58-70fc-4069-bdba-f4726c87e4d2 nodeName:}" failed. No retries permitted until 2026-07-07 17:40:06.148671427 +0000 UTC m=+10.187998336 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs") pod "network-metrics-daemon-jdmfc" (UID: "9541dd58-70fc-4069-bdba-f4726c87e4d2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:40:02.149388 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:02.149293 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:40:02.149388 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:02.149360 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret podName:970a1d18-134b-417d-8b58-aa2ddd03d5bf nodeName:}" failed. No retries permitted until 2026-07-07 17:40:06.149341735 +0000 UTC m=+10.188668625 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret") pod "global-pull-secret-syncer-q4psm" (UID: "970a1d18-134b-417d-8b58-aa2ddd03d5bf") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:40:02.250226 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:02.249994 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cw84\" (UniqueName: \"kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84\") pod \"network-check-target-qtftl\" (UID: \"4f7c694f-4fcf-4310-9772-12030534d40a\") " pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:02.250226 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:02.250212 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:40:02.250226 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:02.250239 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:40:02.250628 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:02.250253 2577 projected.go:196] Error preparing data for projected volume kube-api-access-4cw84 for pod openshift-network-diagnostics/network-check-target-qtftl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:40:02.250628 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:02.250322 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84 podName:4f7c694f-4fcf-4310-9772-12030534d40a nodeName:}" failed. No retries permitted until 2026-07-07 17:40:06.250301369 +0000 UTC m=+10.289628254 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4cw84" (UniqueName: "kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84") pod "network-check-target-qtftl" (UID: "4f7c694f-4fcf-4310-9772-12030534d40a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:40:02.511323 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:02.511231 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:02.511812 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:02.511378 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:02.511903 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:02.511881 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:02.512035 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:02.512012 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:03.511061 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:03.511001 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:03.511245 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:03.511185 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:04.523336 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:04.523300 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:04.523862 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:04.523300 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:04.523862 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:04.523512 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:04.523862 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:04.523676 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:05.510685 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:05.510127 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:05.510685 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:05.510279 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:06.184925 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:06.184888 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:06.185482 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:06.184950 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs\") pod \"network-metrics-daemon-jdmfc\" (UID: \"9541dd58-70fc-4069-bdba-f4726c87e4d2\") " pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:06.185482 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:06.185105 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:40:06.185482 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:06.185105 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:40:06.185482 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:06.185181 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret podName:970a1d18-134b-417d-8b58-aa2ddd03d5bf nodeName:}" failed. No retries permitted until 2026-07-07 17:40:14.185165507 +0000 UTC m=+18.224492389 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret") pod "global-pull-secret-syncer-q4psm" (UID: "970a1d18-134b-417d-8b58-aa2ddd03d5bf") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:40:06.185482 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:06.185200 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs podName:9541dd58-70fc-4069-bdba-f4726c87e4d2 nodeName:}" failed. No retries permitted until 2026-07-07 17:40:14.185191949 +0000 UTC m=+18.224518832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs") pod "network-metrics-daemon-jdmfc" (UID: "9541dd58-70fc-4069-bdba-f4726c87e4d2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:40:06.286049 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:06.286003 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cw84\" (UniqueName: \"kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84\") pod \"network-check-target-qtftl\" (UID: \"4f7c694f-4fcf-4310-9772-12030534d40a\") " pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:06.286732 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:06.286226 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:40:06.286732 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:06.286262 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:40:06.286732 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:06.286277 2577 projected.go:196] Error preparing data for projected volume kube-api-access-4cw84 for pod openshift-network-diagnostics/network-check-target-qtftl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:40:06.286732 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:06.286345 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84 podName:4f7c694f-4fcf-4310-9772-12030534d40a nodeName:}" failed. No retries permitted until 2026-07-07 17:40:14.286323505 +0000 UTC m=+18.325650391 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4cw84" (UniqueName: "kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84") pod "network-check-target-qtftl" (UID: "4f7c694f-4fcf-4310-9772-12030534d40a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:40:06.512083 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:06.511344 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:06.512083 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:06.511469 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:06.512083 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:06.511867 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:06.512083 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:06.511981 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:07.509983 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:07.509936 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:07.510460 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:07.510093 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:08.512974 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:08.512082 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:08.512974 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:08.512240 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:08.512974 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:08.512755 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:08.512974 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:08.512913 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:09.510602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:09.510571 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:09.510734 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:09.510700 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:10.510373 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:10.510337 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:10.510838 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:10.510483 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:10.510979 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:10.510958 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:10.511150 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:10.511119 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:11.510899 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:11.510708 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:11.512810 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:11.511503 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:11.578585 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:11.578546 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" event={"ID":"76382e09-f5ec-4ea9-8bfc-42a4a31c2638","Type":"ContainerStarted","Data":"aa013cefce62d31ec51e63db2bde30e40919ddabb486f202536a77f52ac1fc1b"} Jul 07 17:40:11.581168 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:11.581124 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-170.ec2.internal" event={"ID":"0973ca2b89d42fa29ef5474b01a07c7d","Type":"ContainerStarted","Data":"b4041206b55538a01bba98acd410b30d812b0cbb8ef81683cdb1bef9a0c7bc06"} Jul 07 17:40:11.613277 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:11.613221 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-170.ec2.internal" podStartSLOduration=13.613202337 podStartE2EDuration="13.613202337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:39:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:40:11.613102559 +0000 UTC m=+15.652429462" watchObservedRunningTime="2026-07-07 17:40:11.613202337 +0000 UTC m=+15.652529430" Jul 07 17:40:11.613453 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:11.613300 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9rnhd" podStartSLOduration=2.703037968 podStartE2EDuration="14.613295448s" totalImagesPullingTime="11.91025748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:40:11.597596229 +0000 UTC m=+15.636923124" watchObservedRunningTime="2026-07-07 17:40:11.613295448 +0000 UTC m=+15.652622352" Jul 07 17:40:12.510285 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:12.510018 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:12.510470 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:12.510020 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:12.510470 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:12.510401 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:12.510470 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:12.510460 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:12.585116 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:12.585064 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal" event={"ID":"7c76a1f86a523bd2550ec024c98736a4","Type":"ContainerStarted","Data":"b6a0eaa52beaa2762721802a34ad431f477898e748b50f291f10cb58a582d4bb"} Jul 07 17:40:12.586901 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:12.586846 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw276" event={"ID":"557c3dc0-235b-4285-ad82-77d2fbbfa75f","Type":"ContainerStarted","Data":"acf990de70f0b775c3b59afdce24be333fcb9f52e94715536813dbbc8f28f7f5"} Jul 07 17:40:12.588571 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:12.588538 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" event={"ID":"e7f9d035-66c6-4888-89af-15077571b05f","Type":"ContainerStarted","Data":"5547d68961e60056bb1af60aa5112da1336da829c4e8d188baa80d3d02635d4c"} Jul 07 17:40:12.590581 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:12.590195 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qrfcq" event={"ID":"cb7150e3-b3f8-4086-883c-2a2d4720246b","Type":"ContainerStarted","Data":"92fb4e83e58314a9bf4d60330a215e6cb57371481965c19ef4d1f5e4f48aee44"} Jul 07 17:40:12.592024 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:12.591994 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t5q9p" event={"ID":"4db38d2b-e752-4fdf-81ed-d134f7831397","Type":"ContainerStarted","Data":"d6df97f652325a90bef2adfcbc376d0d5b00015e9b0a16432ace6499927dca86"} Jul 07 17:40:12.593838 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:12.593807 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nck8x" event={"ID":"d1d8d149-d9ff-4b3c-851f-ec9de53109b6","Type":"ContainerStarted","Data":"397e4de80b0a8368a885749e7b7c3b8a05f6b1967b702a2a133e2bb5fccd6014"} Jul 07 17:40:12.595530 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:12.595496 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n4rvd" event={"ID":"6d1fec5d-b6fb-44ac-bd0d-341cfd7c56c8","Type":"ContainerStarted","Data":"2d14b937851bab97a9c4ac0b3f826c651e444b36a7d5e23a4e592534917b3bf4"} Jul 07 17:40:12.614280 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:12.614181 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-t5q9p" podStartSLOduration=3.682844764 podStartE2EDuration="15.614158035s" totalImagesPullingTime="11.931313271s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:40:12.613638935 +0000 UTC m=+16.652965842" watchObservedRunningTime="2026-07-07 17:40:12.614158035 +0000 UTC m=+16.653484940" Jul 07 17:40:12.629593 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:12.629530 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-n4rvd" podStartSLOduration=3.705654463 podStartE2EDuration="15.629505875s" totalImagesPullingTime="11.923851412s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:40:12.628960681 +0000 UTC m=+16.668287584" watchObservedRunningTime="2026-07-07 17:40:12.629505875 +0000 UTC m=+16.668832778" Jul 07 17:40:12.683783 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:12.683729 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qrfcq" podStartSLOduration=3.776621095 podStartE2EDuration="15.683711883s" totalImagesPullingTime="11.907090788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:40:12.668510058 +0000 UTC m=+16.707836960" watchObservedRunningTime="2026-07-07 17:40:12.683711883 +0000 UTC m=+16.723038785" Jul 07 17:40:12.683950 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:12.683915 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nck8x" podStartSLOduration=3.790222939 podStartE2EDuration="15.683907063s" totalImagesPullingTime="11.893684124s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:40:12.683593221 +0000 UTC m=+16.722920125" watchObservedRunningTime="2026-07-07 17:40:12.683907063 +0000 UTC m=+16.723233969" Jul 07 17:40:13.510976 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:13.510940 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:13.511179 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:13.511079 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:14.256165 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:14.256127 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:14.256783 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:14.256195 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs\") pod \"network-metrics-daemon-jdmfc\" (UID: \"9541dd58-70fc-4069-bdba-f4726c87e4d2\") " pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:14.256783 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:14.256308 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:40:14.256783 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:14.256397 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs podName:9541dd58-70fc-4069-bdba-f4726c87e4d2 nodeName:}" failed. No retries permitted until 2026-07-07 17:40:30.256374903 +0000 UTC m=+34.295701788 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs") pod "network-metrics-daemon-jdmfc" (UID: "9541dd58-70fc-4069-bdba-f4726c87e4d2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:40:14.256783 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:14.256308 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:40:14.256783 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:14.256479 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret podName:970a1d18-134b-417d-8b58-aa2ddd03d5bf nodeName:}" failed. No retries permitted until 2026-07-07 17:40:30.256460474 +0000 UTC m=+34.295787373 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret") pod "global-pull-secret-syncer-q4psm" (UID: "970a1d18-134b-417d-8b58-aa2ddd03d5bf") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:40:14.356812 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:14.356771 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cw84\" (UniqueName: \"kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84\") pod \"network-check-target-qtftl\" (UID: \"4f7c694f-4fcf-4310-9772-12030534d40a\") " pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:14.357035 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:14.356937 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:40:14.357035 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:14.356958 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:40:14.357035 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:14.356971 2577 projected.go:196] Error preparing data for projected volume kube-api-access-4cw84 for pod openshift-network-diagnostics/network-check-target-qtftl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:40:14.357035 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:14.357055 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84 podName:4f7c694f-4fcf-4310-9772-12030534d40a nodeName:}" failed. No retries permitted until 2026-07-07 17:40:30.357020065 +0000 UTC m=+34.396346959 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4cw84" (UniqueName: "kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84") pod "network-check-target-qtftl" (UID: "4f7c694f-4fcf-4310-9772-12030534d40a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:40:14.512228 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:14.512147 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:14.512228 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:14.512178 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:14.512452 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:14.512279 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:14.512452 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:14.512367 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:14.692899 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:14.692862 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qrfcq" Jul 07 17:40:14.693511 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:14.693486 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qrfcq" Jul 07 17:40:15.510920 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:15.510887 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:15.511553 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:15.511013 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:15.603381 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:15.603344 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qrfcq" Jul 07 17:40:15.603898 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:15.603874 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qrfcq" Jul 07 17:40:16.511263 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:16.511222 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:16.511901 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:16.511348 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:16.511901 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:16.511393 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:16.511901 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:16.511513 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:17.510352 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:17.510317 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:17.510551 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:17.510459 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:18.510447 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:18.510411 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:18.510983 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:18.510539 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:18.510983 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:18.510905 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:18.511090 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:18.511005 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:19.510498 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:19.510459 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:19.510938 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:19.510591 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:20.513076 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:20.512328 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:20.513076 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:20.512537 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:20.513076 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:20.512725 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:20.513076 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:20.512829 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:20.617573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:20.617524 2577 generic.go:356] "Generic (PLEG): container finished" podID="7c76a1f86a523bd2550ec024c98736a4" containerID="b6a0eaa52beaa2762721802a34ad431f477898e748b50f291f10cb58a582d4bb" exitCode=0 Jul 07 17:40:20.617763 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:20.617597 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal" event={"ID":"7c76a1f86a523bd2550ec024c98736a4","Type":"ContainerDied","Data":"b6a0eaa52beaa2762721802a34ad431f477898e748b50f291f10cb58a582d4bb"} Jul 07 17:40:20.619317 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:20.619276 2577 generic.go:356] "Generic (PLEG): container finished" podID="557c3dc0-235b-4285-ad82-77d2fbbfa75f" containerID="acf990de70f0b775c3b59afdce24be333fcb9f52e94715536813dbbc8f28f7f5" exitCode=0 Jul 07 17:40:20.619441 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:20.619327 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw276" event={"ID":"557c3dc0-235b-4285-ad82-77d2fbbfa75f","Type":"ContainerDied","Data":"acf990de70f0b775c3b59afdce24be333fcb9f52e94715536813dbbc8f28f7f5"} Jul 07 17:40:20.835637 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:20.835588 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 17:40:21.476857 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.476732 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T17:40:20.835605278Z","UUID":"3a427f42-afb7-439c-b517-fb844450a4b4","Handler":null,"Name":"","Endpoint":""} Jul 07 17:40:21.479254 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.479221 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 17:40:21.479431 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.479265 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 17:40:21.510757 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.510721 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:21.510933 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:21.510855 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:21.630969 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.630919 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" event={"ID":"2fcb0730-da1f-4113-aa11-4232d0c7dbf8","Type":"ContainerStarted","Data":"ab1d1b56a2651729e1c7728359ff6b06cc8823de319a0e808096c7a6d721a82a"} Jul 07 17:40:21.631597 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.631070 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" event={"ID":"2fcb0730-da1f-4113-aa11-4232d0c7dbf8","Type":"ContainerStarted","Data":"30f173a5937dbd4be9f419280dab86f3e019ca842b57af534802f829d8e3b3a0"} Jul 07 17:40:21.631597 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.631115 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" event={"ID":"2fcb0730-da1f-4113-aa11-4232d0c7dbf8","Type":"ContainerStarted","Data":"a3c95ecaa58c58d386c14e470cdb69aa3be37827f969545030efe0d966ee4cb8"} Jul 07 17:40:21.631597 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.631165 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" event={"ID":"2fcb0730-da1f-4113-aa11-4232d0c7dbf8","Type":"ContainerStarted","Data":"1bbfd6f775678604f0d30977be619f5f2975c04bf2158283a5a59103a1cadb85"} Jul 07 17:40:21.631597 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.631214 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" event={"ID":"2fcb0730-da1f-4113-aa11-4232d0c7dbf8","Type":"ContainerStarted","Data":"9cd434eb949d3ff8dc106ef9fc5fcb3e5de2e9afb205934bdd767d12ea124ed1"} Jul 07 17:40:21.631597 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.631260 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" event={"ID":"2fcb0730-da1f-4113-aa11-4232d0c7dbf8","Type":"ContainerStarted","Data":"df911b3dd572a4ea256afa653c2acc8741b8c68e1f4eff0cf3e3a3382120c422"} Jul 07 17:40:21.632991 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.632961 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6vdhw" event={"ID":"7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71","Type":"ContainerStarted","Data":"168b72b2699c3948e862893d03799331161436039c8d726134d0ddd05c75f07f"} Jul 07 17:40:21.635341 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.635311 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal" event={"ID":"7c76a1f86a523bd2550ec024c98736a4","Type":"ContainerStarted","Data":"6f1841366e2137daae82d85ab004fa887b2a087fd302712d314689db3c1d8e5f"} Jul 07 17:40:21.637438 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.637403 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" event={"ID":"e7f9d035-66c6-4888-89af-15077571b05f","Type":"ContainerStarted","Data":"26ce59893a462133730cedb6b44b1d0f43f15c9604ee1ef42d354a154c8c1abb"} Jul 07 17:40:21.663100 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.663015 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6vdhw" podStartSLOduration=2.79178944 podStartE2EDuration="24.66300151s" totalImagesPullingTime="21.87121207s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:40:21.662546642 +0000 UTC m=+25.701873544" watchObservedRunningTime="2026-07-07 17:40:21.66300151 +0000 UTC m=+25.702328410" Jul 07 17:40:21.684321 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:21.684266 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-170.ec2.internal" podStartSLOduration=23.68425014 podStartE2EDuration="23.68425014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:39:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:40:21.683927507 +0000 UTC m=+25.723254411" watchObservedRunningTime="2026-07-07 17:40:21.68425014 +0000 UTC m=+25.723577043" Jul 07 17:40:22.510982 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:22.510942 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:22.510982 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:22.510989 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:22.511198 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:22.511103 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:22.511243 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:22.511225 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:22.642279 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:22.642190 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" event={"ID":"e7f9d035-66c6-4888-89af-15077571b05f","Type":"ContainerStarted","Data":"7f395fa67f164599b10ff7574d6387de63c0de37da861a53b445fef2f27fdc14"} Jul 07 17:40:22.662834 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:22.662776 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nnwjr" podStartSLOduration=3.7550592959999998 podStartE2EDuration="25.662758367s" totalImagesPullingTime="21.907699071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:39:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:40:22.66180989 +0000 UTC m=+26.701136794" watchObservedRunningTime="2026-07-07 17:40:22.662758367 +0000 UTC m=+26.702085269" Jul 07 17:40:23.510947 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:23.510911 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:23.511151 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:23.511020 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:23.648976 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:23.648929 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" event={"ID":"2fcb0730-da1f-4113-aa11-4232d0c7dbf8","Type":"ContainerStarted","Data":"f6afb80e297afe506b558e67b5bffb77eff8e01959619c63a7164d633e21c0c9"} Jul 07 17:40:24.510351 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:24.510309 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:24.510549 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:24.510367 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:24.510549 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:24.510476 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:24.510664 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:24.510591 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:25.510299 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:25.510264 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:25.511116 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:25.510401 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:25.660928 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:25.660509 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" event={"ID":"2fcb0730-da1f-4113-aa11-4232d0c7dbf8","Type":"ContainerStarted","Data":"84959cfb41d64bc5ed9ada433e0da732a1aeadd4bf787647de7df9fe8a679f68"} Jul 07 17:40:25.660928 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:25.660753 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:40:25.660928 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:25.660805 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:40:25.680279 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:25.680251 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:40:25.694507 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:25.694455 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" podStartSLOduration=6.780671883 podStartE2EDuration="28.694437575s" totalImagesPullingTime="21.913765692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:40:25.693708657 +0000 UTC m=+29.733035561" watchObservedRunningTime="2026-07-07 17:40:25.694437575 +0000 UTC m=+29.733764481" Jul 07 17:40:26.510995 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:26.510948 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:26.511478 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:26.511119 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:26.511478 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:26.511176 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:26.511478 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:26.511274 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:26.664021 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:26.663986 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:40:26.682498 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:26.682472 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:40:27.510061 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:27.510019 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:27.510191 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:27.510147 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:27.668332 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:27.668293 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw276" event={"ID":"557c3dc0-235b-4285-ad82-77d2fbbfa75f","Type":"ContainerStarted","Data":"af392cb1d44f0557f6c1979f01302308535dea14425bea76171860caffef36b1"} Jul 07 17:40:27.824604 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:27.824523 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qtftl"] Jul 07 17:40:27.824746 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:27.824649 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:27.824783 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:27.824739 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:27.828584 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:27.828552 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jdmfc"] Jul 07 17:40:27.828727 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:27.828673 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:27.828799 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:27.828781 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:27.830536 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:27.830504 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q4psm"] Jul 07 17:40:27.830640 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:27.830622 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:27.830740 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:27.830721 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:28.672426 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:28.672391 2577 generic.go:356] "Generic (PLEG): container finished" podID="557c3dc0-235b-4285-ad82-77d2fbbfa75f" containerID="af392cb1d44f0557f6c1979f01302308535dea14425bea76171860caffef36b1" exitCode=0 Jul 07 17:40:28.672781 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:28.672466 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw276" event={"ID":"557c3dc0-235b-4285-ad82-77d2fbbfa75f","Type":"ContainerDied","Data":"af392cb1d44f0557f6c1979f01302308535dea14425bea76171860caffef36b1"} Jul 07 17:40:29.510663 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:29.510634 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:29.510759 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:29.510668 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:29.510759 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:29.510741 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:29.510867 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:29.510756 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:29.510920 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:29.510856 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:29.510973 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:29.510945 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:29.677541 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:29.677354 2577 generic.go:356] "Generic (PLEG): container finished" podID="557c3dc0-235b-4285-ad82-77d2fbbfa75f" containerID="c937a3b50e8725f5496d6bb14442c1c4f900311025b49f50bcc932580255ceeb" exitCode=0 Jul 07 17:40:29.677938 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:29.677421 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw276" event={"ID":"557c3dc0-235b-4285-ad82-77d2fbbfa75f","Type":"ContainerDied","Data":"c937a3b50e8725f5496d6bb14442c1c4f900311025b49f50bcc932580255ceeb"} Jul 07 17:40:30.273974 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:30.273938 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:30.274194 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:30.273990 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs\") pod \"network-metrics-daemon-jdmfc\" (UID: \"9541dd58-70fc-4069-bdba-f4726c87e4d2\") " pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:30.274194 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:30.274117 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 17:40:30.274194 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:30.274124 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:40:30.274194 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:30.274189 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret podName:970a1d18-134b-417d-8b58-aa2ddd03d5bf nodeName:}" failed. No retries permitted until 2026-07-07 17:41:02.274174496 +0000 UTC m=+66.313501377 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret") pod "global-pull-secret-syncer-q4psm" (UID: "970a1d18-134b-417d-8b58-aa2ddd03d5bf") : object "kube-system"/"original-pull-secret" not registered Jul 07 17:40:30.274402 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:30.274206 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs podName:9541dd58-70fc-4069-bdba-f4726c87e4d2 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:02.274199055 +0000 UTC m=+66.313525936 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs") pod "network-metrics-daemon-jdmfc" (UID: "9541dd58-70fc-4069-bdba-f4726c87e4d2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 17:40:30.374492 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:30.374444 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cw84\" (UniqueName: \"kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84\") pod \"network-check-target-qtftl\" (UID: \"4f7c694f-4fcf-4310-9772-12030534d40a\") " pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:30.374668 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:30.374604 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 17:40:30.374668 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:30.374627 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 17:40:30.374668 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:30.374637 2577 projected.go:196] Error preparing data for projected volume kube-api-access-4cw84 for pod openshift-network-diagnostics/network-check-target-qtftl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:40:30.374760 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:30.374692 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84 podName:4f7c694f-4fcf-4310-9772-12030534d40a nodeName:}" failed. No retries permitted until 2026-07-07 17:41:02.374675466 +0000 UTC m=+66.414002347 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4cw84" (UniqueName: "kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84") pod "network-check-target-qtftl" (UID: "4f7c694f-4fcf-4310-9772-12030534d40a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 17:40:31.510486 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:31.510446 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:31.511067 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:31.510494 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:31.511067 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:31.510496 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:31.511067 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:31.510574 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q4psm" podUID="970a1d18-134b-417d-8b58-aa2ddd03d5bf" Jul 07 17:40:31.511067 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:31.510657 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jdmfc" podUID="9541dd58-70fc-4069-bdba-f4726c87e4d2" Jul 07 17:40:31.511067 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:31.510723 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtftl" podUID="4f7c694f-4fcf-4310-9772-12030534d40a" Jul 07 17:40:31.687504 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:31.687465 2577 generic.go:356] "Generic (PLEG): container finished" podID="557c3dc0-235b-4285-ad82-77d2fbbfa75f" containerID="1ccd616e75b1ef2cf64a6d689943d376df1ffbd78f8557996f6ecfe162c9af23" exitCode=0 Jul 07 17:40:31.687746 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:31.687525 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw276" event={"ID":"557c3dc0-235b-4285-ad82-77d2fbbfa75f","Type":"ContainerDied","Data":"1ccd616e75b1ef2cf64a6d689943d376df1ffbd78f8557996f6ecfe162c9af23"} Jul 07 17:40:33.252900 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.252872 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-170.ec2.internal" event="NodeReady" Jul 07 17:40:33.253523 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.253026 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 17:40:33.297343 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.297307 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-675994669b-7mkgg"] Jul 07 17:40:33.309216 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.309186 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.313797 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.313776 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 17:40:33.314278 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.314263 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 17:40:33.314474 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.314456 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vx5vk\"" Jul 07 17:40:33.314731 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.314709 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 17:40:33.316620 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.316600 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b2xkh"] Jul 07 17:40:33.321946 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.321872 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 17:40:33.324169 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.324138 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-x5q45"] Jul 07 17:40:33.324329 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.324244 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:33.328162 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.327875 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 17:40:33.328162 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.327999 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6c56v\"" Jul 07 17:40:33.328383 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.328349 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 17:40:33.332731 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.332690 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-675994669b-7mkgg"] Jul 07 17:40:33.332731 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.332736 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x5q45"] Jul 07 17:40:33.332928 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.332830 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x5q45" Jul 07 17:40:33.335640 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.335613 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 17:40:33.335794 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.335644 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 17:40:33.335794 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.335651 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 17:40:33.336096 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.336077 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8stmj\"" Jul 07 17:40:33.348503 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.348471 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b2xkh"] Jul 07 17:40:33.396190 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.396151 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq5cc\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-kube-api-access-hq5cc\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.396456 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.396208 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:33.396456 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.396241 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mct7\" (UniqueName: \"kubernetes.io/projected/258e1cc3-989f-4b11-9152-31b2aeb13bb9-kube-api-access-9mct7\") pod \"ingress-canary-x5q45\" (UID: \"258e1cc3-989f-4b11-9152-31b2aeb13bb9\") " pod="openshift-ingress-canary/ingress-canary-x5q45" Jul 07 17:40:33.396456 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.396267 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.396456 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.396292 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbpzz\" (UniqueName: \"kubernetes.io/projected/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-kube-api-access-kbpzz\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:33.396456 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.396363 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/89619451-c733-4743-90b3-ec2cffa338fc-ca-trust-extracted\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.396456 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.396396 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/89619451-c733-4743-90b3-ec2cffa338fc-image-registry-private-configuration\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.396456 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.396422 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/89619451-c733-4743-90b3-ec2cffa338fc-installation-pull-secrets\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.396456 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.396458 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/89619451-c733-4743-90b3-ec2cffa338fc-trusted-ca\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.397012 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.396527 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/258e1cc3-989f-4b11-9152-31b2aeb13bb9-cert\") pod \"ingress-canary-x5q45\" (UID: \"258e1cc3-989f-4b11-9152-31b2aeb13bb9\") " pod="openshift-ingress-canary/ingress-canary-x5q45" Jul 07 17:40:33.397012 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.396628 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/89619451-c733-4743-90b3-ec2cffa338fc-registry-certificates\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.397012 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.396664 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-tmp-dir\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:33.397012 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.396720 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-bound-sa-token\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.397012 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.396762 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-config-volume\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:33.497377 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497341 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/89619451-c733-4743-90b3-ec2cffa338fc-ca-trust-extracted\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.497377 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497384 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/89619451-c733-4743-90b3-ec2cffa338fc-image-registry-private-configuration\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.497660 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497405 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/89619451-c733-4743-90b3-ec2cffa338fc-installation-pull-secrets\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.497660 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497429 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/89619451-c733-4743-90b3-ec2cffa338fc-trusted-ca\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.497660 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497455 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/258e1cc3-989f-4b11-9152-31b2aeb13bb9-cert\") pod \"ingress-canary-x5q45\" (UID: \"258e1cc3-989f-4b11-9152-31b2aeb13bb9\") " pod="openshift-ingress-canary/ingress-canary-x5q45" Jul 07 17:40:33.497660 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497500 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/89619451-c733-4743-90b3-ec2cffa338fc-registry-certificates\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.497660 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497522 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-tmp-dir\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:33.497660 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497549 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-bound-sa-token\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.497660 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497584 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-config-volume\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:33.497660 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497623 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq5cc\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-kube-api-access-hq5cc\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.497660 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497644 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:33.497965 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497669 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mct7\" (UniqueName: \"kubernetes.io/projected/258e1cc3-989f-4b11-9152-31b2aeb13bb9-kube-api-access-9mct7\") pod \"ingress-canary-x5q45\" (UID: \"258e1cc3-989f-4b11-9152-31b2aeb13bb9\") " pod="openshift-ingress-canary/ingress-canary-x5q45" Jul 07 17:40:33.497965 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497699 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.497965 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497722 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbpzz\" (UniqueName: \"kubernetes.io/projected/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-kube-api-access-kbpzz\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:33.497965 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.497886 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/89619451-c733-4743-90b3-ec2cffa338fc-ca-trust-extracted\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.498233 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:33.498217 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:40:33.498295 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:33.498284 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls podName:8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71 nodeName:}" failed. No retries permitted until 2026-07-07 17:40:33.99826931 +0000 UTC m=+38.037596193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls") pod "dns-default-b2xkh" (UID: "8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71") : secret "dns-default-metrics-tls" not found Jul 07 17:40:33.498364 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:33.498338 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:40:33.498364 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:33.498346 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-675994669b-7mkgg: secret "image-registry-tls" not found Jul 07 17:40:33.498448 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:33.498367 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls podName:89619451-c733-4743-90b3-ec2cffa338fc nodeName:}" failed. No retries permitted until 2026-07-07 17:40:33.998360622 +0000 UTC m=+38.037687506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls") pod "image-registry-675994669b-7mkgg" (UID: "89619451-c733-4743-90b3-ec2cffa338fc") : secret "image-registry-tls" not found Jul 07 17:40:33.498534 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.498512 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-config-volume\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:33.498587 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.498573 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-tmp-dir\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:33.499550 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.499521 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/89619451-c733-4743-90b3-ec2cffa338fc-trusted-ca\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.499951 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.499882 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/89619451-c733-4743-90b3-ec2cffa338fc-registry-certificates\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.503122 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.503091 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/89619451-c733-4743-90b3-ec2cffa338fc-installation-pull-secrets\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.503269 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.503098 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/258e1cc3-989f-4b11-9152-31b2aeb13bb9-cert\") pod \"ingress-canary-x5q45\" (UID: \"258e1cc3-989f-4b11-9152-31b2aeb13bb9\") " pod="openshift-ingress-canary/ingress-canary-x5q45" Jul 07 17:40:33.503269 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.503098 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/89619451-c733-4743-90b3-ec2cffa338fc-image-registry-private-configuration\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.507842 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.507776 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbpzz\" (UniqueName: \"kubernetes.io/projected/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-kube-api-access-kbpzz\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:33.508004 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.507898 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-bound-sa-token\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.508480 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.508456 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq5cc\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-kube-api-access-hq5cc\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:33.508568 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.508461 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mct7\" (UniqueName: \"kubernetes.io/projected/258e1cc3-989f-4b11-9152-31b2aeb13bb9-kube-api-access-9mct7\") pod \"ingress-canary-x5q45\" (UID: \"258e1cc3-989f-4b11-9152-31b2aeb13bb9\") " pod="openshift-ingress-canary/ingress-canary-x5q45" Jul 07 17:40:33.510296 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.510269 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:40:33.510639 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.510621 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:40:33.510810 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.510790 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:40:33.512940 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.512917 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 17:40:33.513095 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.512952 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 17:40:33.513095 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.512999 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pvl4h\"" Jul 07 17:40:33.513254 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.513230 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 17:40:33.513371 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.513356 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 17:40:33.513420 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.513382 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2hwnn\"" Jul 07 17:40:33.643536 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.643443 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x5q45" Jul 07 17:40:33.862709 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:33.862448 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x5q45"] Jul 07 17:40:33.867503 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:40:33.867469 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod258e1cc3_989f_4b11_9152_31b2aeb13bb9.slice/crio-3c68c115e99f37665b56dba9e279710eee5a7822be25a6e14764b4986868a46f WatchSource:0}: Error finding container 3c68c115e99f37665b56dba9e279710eee5a7822be25a6e14764b4986868a46f: Status 404 returned error can't find the container with id 3c68c115e99f37665b56dba9e279710eee5a7822be25a6e14764b4986868a46f Jul 07 17:40:34.002275 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:34.002236 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:34.002512 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:34.002288 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:34.002512 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:34.002412 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:40:34.002512 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:34.002425 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:40:34.002512 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:34.002444 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-675994669b-7mkgg: secret "image-registry-tls" not found Jul 07 17:40:34.002512 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:34.002498 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls podName:8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71 nodeName:}" failed. No retries permitted until 2026-07-07 17:40:35.002476133 +0000 UTC m=+39.041803019 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls") pod "dns-default-b2xkh" (UID: "8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71") : secret "dns-default-metrics-tls" not found Jul 07 17:40:34.002771 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:34.002519 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls podName:89619451-c733-4743-90b3-ec2cffa338fc nodeName:}" failed. No retries permitted until 2026-07-07 17:40:35.002509683 +0000 UTC m=+39.041836570 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls") pod "image-registry-675994669b-7mkgg" (UID: "89619451-c733-4743-90b3-ec2cffa338fc") : secret "image-registry-tls" not found Jul 07 17:40:34.030196 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:34.030157 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:34.167329 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:34.167288 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:34.696279 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:34.696242 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x5q45" event={"ID":"258e1cc3-989f-4b11-9152-31b2aeb13bb9","Type":"ContainerStarted","Data":"3c68c115e99f37665b56dba9e279710eee5a7822be25a6e14764b4986868a46f"} Jul 07 17:40:35.012176 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:35.012078 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:35.012176 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:35.012127 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:35.012399 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:35.012269 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:40:35.012399 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:35.012307 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:40:35.012399 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:35.012323 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-675994669b-7mkgg: secret "image-registry-tls" not found Jul 07 17:40:35.012399 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:35.012361 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls podName:8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71 nodeName:}" failed. No retries permitted until 2026-07-07 17:40:37.012334073 +0000 UTC m=+41.051660974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls") pod "dns-default-b2xkh" (UID: "8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71") : secret "dns-default-metrics-tls" not found Jul 07 17:40:35.012399 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:35.012386 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls podName:89619451-c733-4743-90b3-ec2cffa338fc nodeName:}" failed. No retries permitted until 2026-07-07 17:40:37.012375239 +0000 UTC m=+41.051702125 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls") pod "image-registry-675994669b-7mkgg" (UID: "89619451-c733-4743-90b3-ec2cffa338fc") : secret "image-registry-tls" not found Jul 07 17:40:37.031245 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:37.031197 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:37.031707 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:37.031263 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:37.031707 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:37.031350 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:40:37.031707 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:37.031428 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:40:37.031707 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:37.031447 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-675994669b-7mkgg: secret "image-registry-tls" not found Jul 07 17:40:37.031707 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:37.031434 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls podName:8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71 nodeName:}" failed. No retries permitted until 2026-07-07 17:40:41.031416844 +0000 UTC m=+45.070743729 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls") pod "dns-default-b2xkh" (UID: "8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71") : secret "dns-default-metrics-tls" not found Jul 07 17:40:37.031707 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:37.031511 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls podName:89619451-c733-4743-90b3-ec2cffa338fc nodeName:}" failed. No retries permitted until 2026-07-07 17:40:41.031496008 +0000 UTC m=+45.070822889 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls") pod "image-registry-675994669b-7mkgg" (UID: "89619451-c733-4743-90b3-ec2cffa338fc") : secret "image-registry-tls" not found Jul 07 17:40:39.299925 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:39.299656 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:39.365334 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:39.365290 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:39.448805 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:39.448770 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:39.526354 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:39.526310 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:39.608392 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:39.608306 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:39.685302 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:39.685262 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:39.710783 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:39.710749 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x5q45" event={"ID":"258e1cc3-989f-4b11-9152-31b2aeb13bb9","Type":"ContainerStarted","Data":"a20afb68893816a24b9f867bb7a596a838bba126fc06cc86eeaed61b2edbb028"} Jul 07 17:40:39.713845 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:39.713810 2577 generic.go:356] "Generic (PLEG): container finished" podID="557c3dc0-235b-4285-ad82-77d2fbbfa75f" containerID="bae6052b087eaf41ec70fc0a0117358072f4a876e4d410645d62c321bdf7f625" exitCode=0 Jul 07 17:40:39.714013 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:39.713874 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw276" event={"ID":"557c3dc0-235b-4285-ad82-77d2fbbfa75f","Type":"ContainerDied","Data":"bae6052b087eaf41ec70fc0a0117358072f4a876e4d410645d62c321bdf7f625"} Jul 07 17:40:39.726463 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:39.726412 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-x5q45" podStartSLOduration=1.892459756 podStartE2EDuration="6.726397093s" totalImagesPullingTime="4.833937337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:40:39.725486705 +0000 UTC m=+43.764813607" watchObservedRunningTime="2026-07-07 17:40:39.726397093 +0000 UTC m=+43.765724030" Jul 07 17:40:40.340235 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:40.340194 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:40.423164 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:40.423124 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:40.719502 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:40.719471 2577 generic.go:356] "Generic (PLEG): container finished" podID="557c3dc0-235b-4285-ad82-77d2fbbfa75f" containerID="d37b3ea7a8af1590432b83c024c2ec772c22bd8c03bbc27e1cc62b62a3264126" exitCode=0 Jul 07 17:40:40.719668 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:40.719538 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw276" event={"ID":"557c3dc0-235b-4285-ad82-77d2fbbfa75f","Type":"ContainerDied","Data":"d37b3ea7a8af1590432b83c024c2ec772c22bd8c03bbc27e1cc62b62a3264126"} Jul 07 17:40:40.733195 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:40.733162 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:40.925067 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:40.925011 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:41.068162 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:41.068064 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:41.068162 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:41.068115 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:41.068353 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:41.068224 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:40:41.068353 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:41.068222 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:40:41.068353 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:41.068305 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls podName:8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71 nodeName:}" failed. No retries permitted until 2026-07-07 17:40:49.068288628 +0000 UTC m=+53.107615509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls") pod "dns-default-b2xkh" (UID: "8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71") : secret "dns-default-metrics-tls" not found Jul 07 17:40:41.068353 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:41.068236 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-675994669b-7mkgg: secret "image-registry-tls" not found Jul 07 17:40:41.068478 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:41.068385 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls podName:89619451-c733-4743-90b3-ec2cffa338fc nodeName:}" failed. No retries permitted until 2026-07-07 17:40:49.068373035 +0000 UTC m=+53.107699915 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls") pod "image-registry-675994669b-7mkgg" (UID: "89619451-c733-4743-90b3-ec2cffa338fc") : secret "image-registry-tls" not found Jul 07 17:40:41.158185 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:41.158153 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:41.295680 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:41.295634 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:41.392120 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:41.392001 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:41.554415 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:41.554385 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:41.650514 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:41.650417 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:41.724997 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:41.724965 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw276" event={"ID":"557c3dc0-235b-4285-ad82-77d2fbbfa75f","Type":"ContainerStarted","Data":"53189ba4349409198a56e820c86098b03e7b0462b385f2ad7ebf3876a32d8c53"} Jul 07 17:40:41.751154 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:41.751095 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tw276" podStartSLOduration=16.787121494 podStartE2EDuration="44.751076622s" totalImagesPullingTime="27.963955128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:39:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:40:41.749809523 +0000 UTC m=+45.789136426" watchObservedRunningTime="2026-07-07 17:40:41.751076622 +0000 UTC m=+45.790403525" Jul 07 17:40:41.957220 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:41.957174 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:42.095915 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:42.095872 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:42.791328 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:42.791294 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:42.933548 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:42.933515 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:43.254096 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:43.254065 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:43.409024 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:43.408983 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:45.323004 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:45.322970 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:45.423474 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:45.423443 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:49.133945 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:49.133895 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:40:49.133945 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:49.133945 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:40:49.134631 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:49.134068 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:40:49.134631 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:49.134104 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:40:49.134631 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:49.134120 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-675994669b-7mkgg: secret "image-registry-tls" not found Jul 07 17:40:49.134631 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:49.134139 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls podName:8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:05.134124081 +0000 UTC m=+69.173450963 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls") pod "dns-default-b2xkh" (UID: "8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71") : secret "dns-default-metrics-tls" not found Jul 07 17:40:49.134631 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:40:49.134186 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls podName:89619451-c733-4743-90b3-ec2cffa338fc nodeName:}" failed. No retries permitted until 2026-07-07 17:41:05.134169872 +0000 UTC m=+69.173496776 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls") pod "image-registry-675994669b-7mkgg" (UID: "89619451-c733-4743-90b3-ec2cffa338fc") : secret "image-registry-tls" not found Jul 07 17:40:52.394880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:52.394836 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:52.457984 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:52.457937 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:56.439812 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:56.439775 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:56.587836 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:56.587791 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" limit=1024 Jul 07 17:40:58.683669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:40:58.683645 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tcrdq" Jul 07 17:41:02.332253 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:02.332195 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs\") pod \"network-metrics-daemon-jdmfc\" (UID: \"9541dd58-70fc-4069-bdba-f4726c87e4d2\") " pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:41:02.332758 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:02.332297 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:41:02.336235 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:02.336213 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 17:41:02.336419 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:02.336407 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 17:41:02.342941 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:02.342905 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 17:41:02.343114 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:02.342984 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs podName:9541dd58-70fc-4069-bdba-f4726c87e4d2 nodeName:}" failed. No retries permitted until 2026-07-07 17:42:06.34296558 +0000 UTC m=+130.382292464 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs") pod "network-metrics-daemon-jdmfc" (UID: "9541dd58-70fc-4069-bdba-f4726c87e4d2") : secret "metrics-daemon-secret" not found Jul 07 17:41:02.347067 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:02.347006 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/970a1d18-134b-417d-8b58-aa2ddd03d5bf-original-pull-secret\") pod \"global-pull-secret-syncer-q4psm\" (UID: \"970a1d18-134b-417d-8b58-aa2ddd03d5bf\") " pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:41:02.432806 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:02.432754 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cw84\" (UniqueName: \"kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84\") pod \"network-check-target-qtftl\" (UID: \"4f7c694f-4fcf-4310-9772-12030534d40a\") " pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:41:02.435569 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:02.435530 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 17:41:02.446058 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:02.446016 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 17:41:02.456412 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:02.456383 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cw84\" (UniqueName: \"kubernetes.io/projected/4f7c694f-4fcf-4310-9772-12030534d40a-kube-api-access-4cw84\") pod \"network-check-target-qtftl\" (UID: \"4f7c694f-4fcf-4310-9772-12030534d40a\") " pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:41:02.633433 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:02.633343 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pvl4h\"" Jul 07 17:41:02.636734 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:02.636695 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q4psm" Jul 07 17:41:02.640053 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:02.640018 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:41:02.808327 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:02.808293 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q4psm"] Jul 07 17:41:02.812225 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:41:02.812193 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod970a1d18_134b_417d_8b58_aa2ddd03d5bf.slice/crio-8ef7b0d90487218909d9b0dac193a5f585d364c68484490175da4609133bc63c WatchSource:0}: Error finding container 8ef7b0d90487218909d9b0dac193a5f585d364c68484490175da4609133bc63c: Status 404 returned error can't find the container with id 8ef7b0d90487218909d9b0dac193a5f585d364c68484490175da4609133bc63c Jul 07 17:41:02.825538 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:02.825507 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qtftl"] Jul 07 17:41:02.829423 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:41:02.829395 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f7c694f_4fcf_4310_9772_12030534d40a.slice/crio-dae9009aac6a74a6c42a26f166afc46b2a9846c5f6333d5eea71f3602a729819 WatchSource:0}: Error finding container dae9009aac6a74a6c42a26f166afc46b2a9846c5f6333d5eea71f3602a729819: Status 404 returned error can't find the container with id dae9009aac6a74a6c42a26f166afc46b2a9846c5f6333d5eea71f3602a729819 Jul 07 17:41:03.784795 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:03.784758 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qtftl" event={"ID":"4f7c694f-4fcf-4310-9772-12030534d40a","Type":"ContainerStarted","Data":"dae9009aac6a74a6c42a26f166afc46b2a9846c5f6333d5eea71f3602a729819"} Jul 07 17:41:03.786098 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:03.786058 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q4psm" event={"ID":"970a1d18-134b-417d-8b58-aa2ddd03d5bf","Type":"ContainerStarted","Data":"8ef7b0d90487218909d9b0dac193a5f585d364c68484490175da4609133bc63c"} Jul 07 17:41:05.155359 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:05.155302 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:41:05.155359 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:05.155361 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:41:05.155839 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:05.155465 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 17:41:05.155839 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:05.155477 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 17:41:05.155839 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:05.155493 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-675994669b-7mkgg: secret "image-registry-tls" not found Jul 07 17:41:05.155839 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:05.155542 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls podName:8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:37.155526878 +0000 UTC m=+101.194853763 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls") pod "dns-default-b2xkh" (UID: "8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71") : secret "dns-default-metrics-tls" not found Jul 07 17:41:05.155839 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:05.155557 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls podName:89619451-c733-4743-90b3-ec2cffa338fc nodeName:}" failed. No retries permitted until 2026-07-07 17:41:37.155550823 +0000 UTC m=+101.194877704 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls") pod "image-registry-675994669b-7mkgg" (UID: "89619451-c733-4743-90b3-ec2cffa338fc") : secret "image-registry-tls" not found Jul 07 17:41:08.801175 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:08.801136 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qtftl" event={"ID":"4f7c694f-4fcf-4310-9772-12030534d40a","Type":"ContainerStarted","Data":"0cff1a0674868444453acb5dd20ed7da9c7effa6bb04d1e6b24cdcf4cf77bcae"} Jul 07 17:41:08.801614 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:08.801211 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:41:08.802692 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:08.802670 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q4psm" event={"ID":"970a1d18-134b-417d-8b58-aa2ddd03d5bf","Type":"ContainerStarted","Data":"ebf47f2252be8b76f3d77dafe1c519e81021dab1c4d526231425d18a4028e823"} Jul 07 17:41:08.821786 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:08.821713 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qtftl" podStartSLOduration=66.51203805 podStartE2EDuration="1m11.821693835s" totalImagesPullingTime="5.309655785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:41:08.82091789 +0000 UTC m=+72.860244834" watchObservedRunningTime="2026-07-07 17:41:08.821693835 +0000 UTC m=+72.861020738" Jul 07 17:41:08.844819 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:08.844759 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-q4psm" podStartSLOduration=66.510954145 podStartE2EDuration="1m11.844743762s" totalImagesPullingTime="5.333789617s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:41:08.844293216 +0000 UTC m=+72.883620130" watchObservedRunningTime="2026-07-07 17:41:08.844743762 +0000 UTC m=+72.884070697" Jul 07 17:41:09.215213 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.215169 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx"] Jul 07 17:41:09.218222 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.218191 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:09.221159 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.221127 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 17:41:09.221311 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.221238 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 17:41:09.223469 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.223436 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 17:41:09.223623 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.223442 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 17:41:09.223623 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.223442 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cjtw6\"" Jul 07 17:41:09.228922 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.228847 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx"] Jul 07 17:41:09.287282 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.287245 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:09.287484 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.287302 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-565x7\" (UniqueName: \"kubernetes.io/projected/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-kube-api-access-565x7\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:09.287484 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.287396 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:09.324135 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.324094 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9"] Jul 07 17:41:09.328117 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.328095 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" Jul 07 17:41:09.329480 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.329440 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz"] Jul 07 17:41:09.332080 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.332060 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-s9wsz"] Jul 07 17:41:09.332177 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.332154 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" Jul 07 17:41:09.332561 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.332464 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 17:41:09.332845 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.332829 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-x6vfq\"" Jul 07 17:41:09.334413 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.334394 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:41:09.334994 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.334977 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-cfnl7\"" Jul 07 17:41:09.335087 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.335022 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 17:41:09.335241 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.335206 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-s9wsz" Jul 07 17:41:09.335318 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.335256 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 07 17:41:09.335379 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.335362 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 07 17:41:09.335585 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.335567 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:41:09.336216 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.336196 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 07 17:41:09.337029 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.337007 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5c5c789b7f-26cs9"] Jul 07 17:41:09.337857 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.337836 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 17:41:09.337934 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.337836 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:41:09.337934 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.337889 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-tltd7\"" Jul 07 17:41:09.339582 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.339563 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-2c8ld"] Jul 07 17:41:09.339705 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.339678 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.342755 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.342726 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 17:41:09.342945 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.342778 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.345753 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.345418 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9"] Jul 07 17:41:09.345753 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.345518 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 17:41:09.347639 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.346753 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 17:41:09.347639 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.346874 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 17:41:09.347639 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.347225 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 17:41:09.347639 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.347370 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jdrfl\"" Jul 07 17:41:09.347639 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.347613 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 17:41:09.348483 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.348457 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 17:41:09.349434 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.349390 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 17:41:09.351323 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.350369 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5xqr9\"" Jul 07 17:41:09.351323 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.350591 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 17:41:09.351323 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.350607 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 17:41:09.357251 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.357221 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz"] Jul 07 17:41:09.362977 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.362233 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 17:41:09.363637 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.363612 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-s9wsz"] Jul 07 17:41:09.364161 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.364136 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5c5c789b7f-26cs9"] Jul 07 17:41:09.372853 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.372822 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-2c8ld"] Jul 07 17:41:09.388123 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388085 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-565x7\" (UniqueName: \"kubernetes.io/projected/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-kube-api-access-565x7\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:09.388123 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388125 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/055ae3eb-05b3-4509-9008-d62baf858fb3-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.388345 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388162 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g4kg\" (UniqueName: \"kubernetes.io/projected/055ae3eb-05b3-4509-9008-d62baf858fb3-kube-api-access-5g4kg\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.388345 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388210 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d707cdcf-3720-438d-a034-cf5dd54f2a47-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-w8smz\" (UID: \"d707cdcf-3720-438d-a034-cf5dd54f2a47\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" Jul 07 17:41:09.388345 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388235 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/055ae3eb-05b3-4509-9008-d62baf858fb3-serving-cert\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.388345 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:09.388345 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388281 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-stats-auth\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.388345 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388306 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-k9wh9\" (UID: \"3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" Jul 07 17:41:09.388345 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388340 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jph8h\" (UniqueName: \"kubernetes.io/projected/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-kube-api-access-jph8h\") pod \"cluster-samples-operator-56fcf86bf-k9wh9\" (UID: \"3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" Jul 07 17:41:09.388674 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388362 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.388674 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388387 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/055ae3eb-05b3-4509-9008-d62baf858fb3-tmp\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.388674 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388439 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhw4c\" (UniqueName: \"kubernetes.io/projected/d707cdcf-3720-438d-a034-cf5dd54f2a47-kube-api-access-nhw4c\") pod \"service-ca-operator-5d46d9ffd4-w8smz\" (UID: \"d707cdcf-3720-438d-a034-cf5dd54f2a47\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" Jul 07 17:41:09.388674 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388471 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/055ae3eb-05b3-4509-9008-d62baf858fb3-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.388674 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388499 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn5ch\" (UniqueName: \"kubernetes.io/projected/344f0c5c-f855-4e9d-acf3-705194c9ade4-kube-api-access-bn5ch\") pod \"volume-data-source-validator-5d456fdbcd-s9wsz\" (UID: \"344f0c5c-f855-4e9d-acf3-705194c9ade4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-s9wsz" Jul 07 17:41:09.388674 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388571 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-default-certificate\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.388674 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388602 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/055ae3eb-05b3-4509-9008-d62baf858fb3-snapshots\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.388674 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388622 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.388896 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388684 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:09.388896 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388736 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d707cdcf-3720-438d-a034-cf5dd54f2a47-config\") pod \"service-ca-operator-5d46d9ffd4-w8smz\" (UID: \"d707cdcf-3720-438d-a034-cf5dd54f2a47\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" Jul 07 17:41:09.388896 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388753 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x99sf\" (UniqueName: \"kubernetes.io/projected/57a355c0-7af9-4412-b621-771fae3a122c-kube-api-access-x99sf\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.388896 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:09.388859 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 17:41:09.389010 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:09.388919 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls podName:2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:09.888906418 +0000 UTC m=+73.928233298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-jzjvx" (UID: "2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4") : secret "cluster-monitoring-operator-tls" not found Jul 07 17:41:09.389010 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.388957 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:09.405938 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.405898 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-565x7\" (UniqueName: \"kubernetes.io/projected/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-kube-api-access-565x7\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:09.431257 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.431221 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4"] Jul 07 17:41:09.454618 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.454587 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4"] Jul 07 17:41:09.454797 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.454689 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" Jul 07 17:41:09.457662 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.457633 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 17:41:09.457662 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.457653 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 17:41:09.457886 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.457662 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 17:41:09.457886 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.457686 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:41:09.457886 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.457636 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-w2z94\"" Jul 07 17:41:09.490108 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.489981 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g4kg\" (UniqueName: \"kubernetes.io/projected/055ae3eb-05b3-4509-9008-d62baf858fb3-kube-api-access-5g4kg\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.490108 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490029 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d707cdcf-3720-438d-a034-cf5dd54f2a47-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-w8smz\" (UID: \"d707cdcf-3720-438d-a034-cf5dd54f2a47\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" Jul 07 17:41:09.490108 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490068 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/055ae3eb-05b3-4509-9008-d62baf858fb3-serving-cert\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.490372 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490189 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxvvr\" (UniqueName: \"kubernetes.io/projected/55437412-4864-499e-82c2-a854b002bb47-kube-api-access-xxvvr\") pod \"kube-storage-version-migrator-operator-846bf668f7-dwjq4\" (UID: \"55437412-4864-499e-82c2-a854b002bb47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" Jul 07 17:41:09.490372 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490231 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-stats-auth\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.490372 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490258 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-k9wh9\" (UID: \"3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" Jul 07 17:41:09.490372 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490293 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jph8h\" (UniqueName: \"kubernetes.io/projected/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-kube-api-access-jph8h\") pod \"cluster-samples-operator-56fcf86bf-k9wh9\" (UID: \"3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" Jul 07 17:41:09.490372 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490319 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.490372 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490342 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/055ae3eb-05b3-4509-9008-d62baf858fb3-tmp\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.490669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490404 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhw4c\" (UniqueName: \"kubernetes.io/projected/d707cdcf-3720-438d-a034-cf5dd54f2a47-kube-api-access-nhw4c\") pod \"service-ca-operator-5d46d9ffd4-w8smz\" (UID: \"d707cdcf-3720-438d-a034-cf5dd54f2a47\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" Jul 07 17:41:09.490669 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:09.490412 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:41:09.490669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490440 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/055ae3eb-05b3-4509-9008-d62baf858fb3-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.490669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490467 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn5ch\" (UniqueName: \"kubernetes.io/projected/344f0c5c-f855-4e9d-acf3-705194c9ade4-kube-api-access-bn5ch\") pod \"volume-data-source-validator-5d456fdbcd-s9wsz\" (UID: \"344f0c5c-f855-4e9d-acf3-705194c9ade4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-s9wsz" Jul 07 17:41:09.490669 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:09.490493 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls podName:3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:09.990469971 +0000 UTC m=+74.029796853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-k9wh9" (UID: "3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32") : secret "samples-operator-tls" not found Jul 07 17:41:09.490669 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:09.490509 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 17:41:09.490669 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:09.490576 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs podName:57a355c0-7af9-4412-b621-771fae3a122c nodeName:}" failed. No retries permitted until 2026-07-07 17:41:09.990555821 +0000 UTC m=+74.029882715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs") pod "router-default-5c5c789b7f-26cs9" (UID: "57a355c0-7af9-4412-b621-771fae3a122c") : secret "router-metrics-certs-default" not found Jul 07 17:41:09.490669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490538 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-default-certificate\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.490669 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490604 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/055ae3eb-05b3-4509-9008-d62baf858fb3-snapshots\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.491135 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490797 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.491135 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490841 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55437412-4864-499e-82c2-a854b002bb47-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-dwjq4\" (UID: \"55437412-4864-499e-82c2-a854b002bb47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" Jul 07 17:41:09.491135 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490911 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d707cdcf-3720-438d-a034-cf5dd54f2a47-config\") pod \"service-ca-operator-5d46d9ffd4-w8smz\" (UID: \"d707cdcf-3720-438d-a034-cf5dd54f2a47\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" Jul 07 17:41:09.491135 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490937 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x99sf\" (UniqueName: \"kubernetes.io/projected/57a355c0-7af9-4412-b621-771fae3a122c-kube-api-access-x99sf\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.491135 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.490971 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/055ae3eb-05b3-4509-9008-d62baf858fb3-tmp\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.491135 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.491027 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/055ae3eb-05b3-4509-9008-d62baf858fb3-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.491135 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.491095 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55437412-4864-499e-82c2-a854b002bb47-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-dwjq4\" (UID: \"55437412-4864-499e-82c2-a854b002bb47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" Jul 07 17:41:09.491532 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:09.491497 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle podName:57a355c0-7af9-4412-b621-771fae3a122c nodeName:}" failed. No retries permitted until 2026-07-07 17:41:09.991477105 +0000 UTC m=+74.030804195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle") pod "router-default-5c5c789b7f-26cs9" (UID: "57a355c0-7af9-4412-b621-771fae3a122c") : configmap references non-existent config key: service-ca.crt Jul 07 17:41:09.491532 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.491506 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/055ae3eb-05b3-4509-9008-d62baf858fb3-snapshots\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.491723 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.491538 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d707cdcf-3720-438d-a034-cf5dd54f2a47-config\") pod \"service-ca-operator-5d46d9ffd4-w8smz\" (UID: \"d707cdcf-3720-438d-a034-cf5dd54f2a47\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" Jul 07 17:41:09.491956 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.491923 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/055ae3eb-05b3-4509-9008-d62baf858fb3-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.492077 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.492008 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/055ae3eb-05b3-4509-9008-d62baf858fb3-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.493090 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.493068 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/055ae3eb-05b3-4509-9008-d62baf858fb3-serving-cert\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.493428 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.493409 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-default-certificate\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.493528 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.493498 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d707cdcf-3720-438d-a034-cf5dd54f2a47-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-w8smz\" (UID: \"d707cdcf-3720-438d-a034-cf5dd54f2a47\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" Jul 07 17:41:09.493704 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.493684 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-stats-auth\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.500444 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.500416 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g4kg\" (UniqueName: \"kubernetes.io/projected/055ae3eb-05b3-4509-9008-d62baf858fb3-kube-api-access-5g4kg\") pod \"insights-operator-5c847dcfc4-2c8ld\" (UID: \"055ae3eb-05b3-4509-9008-d62baf858fb3\") " pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.502892 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.502862 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x99sf\" (UniqueName: \"kubernetes.io/projected/57a355c0-7af9-4412-b621-771fae3a122c-kube-api-access-x99sf\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.503592 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.503569 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn5ch\" (UniqueName: \"kubernetes.io/projected/344f0c5c-f855-4e9d-acf3-705194c9ade4-kube-api-access-bn5ch\") pod \"volume-data-source-validator-5d456fdbcd-s9wsz\" (UID: \"344f0c5c-f855-4e9d-acf3-705194c9ade4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-s9wsz" Jul 07 17:41:09.504313 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.504290 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jph8h\" (UniqueName: \"kubernetes.io/projected/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-kube-api-access-jph8h\") pod \"cluster-samples-operator-56fcf86bf-k9wh9\" (UID: \"3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" Jul 07 17:41:09.506377 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.506354 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhw4c\" (UniqueName: \"kubernetes.io/projected/d707cdcf-3720-438d-a034-cf5dd54f2a47-kube-api-access-nhw4c\") pod \"service-ca-operator-5d46d9ffd4-w8smz\" (UID: \"d707cdcf-3720-438d-a034-cf5dd54f2a47\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" Jul 07 17:41:09.528939 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.528905 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-5brrn"] Jul 07 17:41:09.545505 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.545480 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:09.547554 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.547526 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-5brrn"] Jul 07 17:41:09.550084 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.550060 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 17:41:09.550228 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.550080 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:41:09.550506 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.550480 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 17:41:09.551635 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.551619 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-5h29t\"" Jul 07 17:41:09.554884 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.554862 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 17:41:09.584109 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.584075 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 17:41:09.591566 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.591526 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b473a0e9-cbb7-404f-9f76-94e3bc7394ca-trusted-ca\") pod \"console-operator-68b757fb67-5brrn\" (UID: \"b473a0e9-cbb7-404f-9f76-94e3bc7394ca\") " pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:09.591566 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.591568 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b473a0e9-cbb7-404f-9f76-94e3bc7394ca-serving-cert\") pod \"console-operator-68b757fb67-5brrn\" (UID: \"b473a0e9-cbb7-404f-9f76-94e3bc7394ca\") " pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:09.591813 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.591595 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxvvr\" (UniqueName: \"kubernetes.io/projected/55437412-4864-499e-82c2-a854b002bb47-kube-api-access-xxvvr\") pod \"kube-storage-version-migrator-operator-846bf668f7-dwjq4\" (UID: \"55437412-4864-499e-82c2-a854b002bb47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" Jul 07 17:41:09.591813 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.591688 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2lxj\" (UniqueName: \"kubernetes.io/projected/b473a0e9-cbb7-404f-9f76-94e3bc7394ca-kube-api-access-q2lxj\") pod \"console-operator-68b757fb67-5brrn\" (UID: \"b473a0e9-cbb7-404f-9f76-94e3bc7394ca\") " pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:09.591813 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.591739 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b473a0e9-cbb7-404f-9f76-94e3bc7394ca-config\") pod \"console-operator-68b757fb67-5brrn\" (UID: \"b473a0e9-cbb7-404f-9f76-94e3bc7394ca\") " pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:09.591813 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.591791 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55437412-4864-499e-82c2-a854b002bb47-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-dwjq4\" (UID: \"55437412-4864-499e-82c2-a854b002bb47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" Jul 07 17:41:09.591981 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.591841 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55437412-4864-499e-82c2-a854b002bb47-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-dwjq4\" (UID: \"55437412-4864-499e-82c2-a854b002bb47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" Jul 07 17:41:09.592414 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.592385 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55437412-4864-499e-82c2-a854b002bb47-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-dwjq4\" (UID: \"55437412-4864-499e-82c2-a854b002bb47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" Jul 07 17:41:09.594302 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.594273 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55437412-4864-499e-82c2-a854b002bb47-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-dwjq4\" (UID: \"55437412-4864-499e-82c2-a854b002bb47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" Jul 07 17:41:09.601713 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.601682 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxvvr\" (UniqueName: \"kubernetes.io/projected/55437412-4864-499e-82c2-a854b002bb47-kube-api-access-xxvvr\") pod \"kube-storage-version-migrator-operator-846bf668f7-dwjq4\" (UID: \"55437412-4864-499e-82c2-a854b002bb47\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" Jul 07 17:41:09.649052 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.649011 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" Jul 07 17:41:09.658186 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.658148 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-s9wsz" Jul 07 17:41:09.673009 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.672982 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" Jul 07 17:41:09.693027 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.692990 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b473a0e9-cbb7-404f-9f76-94e3bc7394ca-config\") pod \"console-operator-68b757fb67-5brrn\" (UID: \"b473a0e9-cbb7-404f-9f76-94e3bc7394ca\") " pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:09.693244 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.693218 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b473a0e9-cbb7-404f-9f76-94e3bc7394ca-trusted-ca\") pod \"console-operator-68b757fb67-5brrn\" (UID: \"b473a0e9-cbb7-404f-9f76-94e3bc7394ca\") " pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:09.693315 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.693268 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b473a0e9-cbb7-404f-9f76-94e3bc7394ca-serving-cert\") pod \"console-operator-68b757fb67-5brrn\" (UID: \"b473a0e9-cbb7-404f-9f76-94e3bc7394ca\") " pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:09.693547 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.693512 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2lxj\" (UniqueName: \"kubernetes.io/projected/b473a0e9-cbb7-404f-9f76-94e3bc7394ca-kube-api-access-q2lxj\") pod \"console-operator-68b757fb67-5brrn\" (UID: \"b473a0e9-cbb7-404f-9f76-94e3bc7394ca\") " pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:09.694112 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.693684 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b473a0e9-cbb7-404f-9f76-94e3bc7394ca-config\") pod \"console-operator-68b757fb67-5brrn\" (UID: \"b473a0e9-cbb7-404f-9f76-94e3bc7394ca\") " pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:09.694200 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.694179 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b473a0e9-cbb7-404f-9f76-94e3bc7394ca-trusted-ca\") pod \"console-operator-68b757fb67-5brrn\" (UID: \"b473a0e9-cbb7-404f-9f76-94e3bc7394ca\") " pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:09.696068 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.696010 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b473a0e9-cbb7-404f-9f76-94e3bc7394ca-serving-cert\") pod \"console-operator-68b757fb67-5brrn\" (UID: \"b473a0e9-cbb7-404f-9f76-94e3bc7394ca\") " pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:09.704606 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.704575 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2lxj\" (UniqueName: \"kubernetes.io/projected/b473a0e9-cbb7-404f-9f76-94e3bc7394ca-kube-api-access-q2lxj\") pod \"console-operator-68b757fb67-5brrn\" (UID: \"b473a0e9-cbb7-404f-9f76-94e3bc7394ca\") " pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:09.764687 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.764647 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" Jul 07 17:41:09.832910 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.832862 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz"] Jul 07 17:41:09.835835 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:41:09.835797 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd707cdcf_3720_438d_a034_cf5dd54f2a47.slice/crio-805b8c90481b6e16309b4bb340cbf4692c9aa1ce010e338431fe613f1d64862d WatchSource:0}: Error finding container 805b8c90481b6e16309b4bb340cbf4692c9aa1ce010e338431fe613f1d64862d: Status 404 returned error can't find the container with id 805b8c90481b6e16309b4bb340cbf4692c9aa1ce010e338431fe613f1d64862d Jul 07 17:41:09.839841 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.839809 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-s9wsz"] Jul 07 17:41:09.842806 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:41:09.842775 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod344f0c5c_f855_4e9d_acf3_705194c9ade4.slice/crio-c618985ef182397ca0a6170b6f42b7f1d962610c3e40de4a77bc02c71b45d998 WatchSource:0}: Error finding container c618985ef182397ca0a6170b6f42b7f1d962610c3e40de4a77bc02c71b45d998: Status 404 returned error can't find the container with id c618985ef182397ca0a6170b6f42b7f1d962610c3e40de4a77bc02c71b45d998 Jul 07 17:41:09.856657 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.856623 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:09.865875 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.865703 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-2c8ld"] Jul 07 17:41:09.868671 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:41:09.868634 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod055ae3eb_05b3_4509_9008_d62baf858fb3.slice/crio-ebbde7e25007c185f2b8cb650f1bb0f10649e017f677dc62ab7fab04a085ff42 WatchSource:0}: Error finding container ebbde7e25007c185f2b8cb650f1bb0f10649e017f677dc62ab7fab04a085ff42: Status 404 returned error can't find the container with id ebbde7e25007c185f2b8cb650f1bb0f10649e017f677dc62ab7fab04a085ff42 Jul 07 17:41:09.872523 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.872493 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-8xtch"] Jul 07 17:41:09.894095 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.893534 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-8xtch"] Jul 07 17:41:09.894095 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.893663 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8xtch" Jul 07 17:41:09.895702 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.895665 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:09.896385 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:09.895840 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 17:41:09.896385 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:09.895915 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls podName:2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:10.895894339 +0000 UTC m=+74.935221223 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-jzjvx" (UID: "2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4") : secret "cluster-monitoring-operator-tls" not found Jul 07 17:41:09.896958 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.896785 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-gmccp\"" Jul 07 17:41:09.948845 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.948811 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4"] Jul 07 17:41:09.952746 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:41:09.952709 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55437412_4864_499e_82c2_a854b002bb47.slice/crio-d5987b26e658a54acebc813041d19ea2459f548de6affef263f803d405005270 WatchSource:0}: Error finding container d5987b26e658a54acebc813041d19ea2459f548de6affef263f803d405005270: Status 404 returned error can't find the container with id d5987b26e658a54acebc813041d19ea2459f548de6affef263f803d405005270 Jul 07 17:41:09.996532 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.996441 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.996728 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.996557 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqqjf\" (UniqueName: \"kubernetes.io/projected/f6702dde-6263-479f-933b-dcd73e1ed5ed-kube-api-access-dqqjf\") pod \"network-check-source-989f8b8dc-8xtch\" (UID: \"f6702dde-6263-479f-933b-dcd73e1ed5ed\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8xtch" Jul 07 17:41:09.996728 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.996590 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-k9wh9\" (UID: \"3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" Jul 07 17:41:09.996728 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:09.996612 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle podName:57a355c0-7af9-4412-b621-771fae3a122c nodeName:}" failed. No retries permitted until 2026-07-07 17:41:10.996590072 +0000 UTC m=+75.035916957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle") pod "router-default-5c5c789b7f-26cs9" (UID: "57a355c0-7af9-4412-b621-771fae3a122c") : configmap references non-existent config key: service-ca.crt Jul 07 17:41:09.996728 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:09.996659 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:09.996728 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:09.996683 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:41:09.996728 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:09.996730 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls podName:3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:10.996715639 +0000 UTC m=+75.036042535 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-k9wh9" (UID: "3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32") : secret "samples-operator-tls" not found Jul 07 17:41:09.996946 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:09.996751 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 17:41:09.996946 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:09.996791 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs podName:57a355c0-7af9-4412-b621-771fae3a122c nodeName:}" failed. No retries permitted until 2026-07-07 17:41:10.996782301 +0000 UTC m=+75.036109182 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs") pod "router-default-5c5c789b7f-26cs9" (UID: "57a355c0-7af9-4412-b621-771fae3a122c") : secret "router-metrics-certs-default" not found Jul 07 17:41:10.014231 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:10.014197 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-5brrn"] Jul 07 17:41:10.017622 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:41:10.017593 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb473a0e9_cbb7_404f_9f76_94e3bc7394ca.slice/crio-63e234d0b3508703c55387db20594cffec01c9e7e9411e930a5929400fdf730c WatchSource:0}: Error finding container 63e234d0b3508703c55387db20594cffec01c9e7e9411e930a5929400fdf730c: Status 404 returned error can't find the container with id 63e234d0b3508703c55387db20594cffec01c9e7e9411e930a5929400fdf730c Jul 07 17:41:10.097589 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:10.097548 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqqjf\" (UniqueName: \"kubernetes.io/projected/f6702dde-6263-479f-933b-dcd73e1ed5ed-kube-api-access-dqqjf\") pod \"network-check-source-989f8b8dc-8xtch\" (UID: \"f6702dde-6263-479f-933b-dcd73e1ed5ed\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8xtch" Jul 07 17:41:10.106347 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:10.106316 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqqjf\" (UniqueName: \"kubernetes.io/projected/f6702dde-6263-479f-933b-dcd73e1ed5ed-kube-api-access-dqqjf\") pod \"network-check-source-989f8b8dc-8xtch\" (UID: \"f6702dde-6263-479f-933b-dcd73e1ed5ed\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8xtch" Jul 07 17:41:10.206235 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:10.206197 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8xtch" Jul 07 17:41:10.368153 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:10.368115 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-8xtch"] Jul 07 17:41:10.372100 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:41:10.372070 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6702dde_6263_479f_933b_dcd73e1ed5ed.slice/crio-457b3981b28b06e48a0de672620d6a03805be1f384e4ed3bf3a20f84ca50a76e WatchSource:0}: Error finding container 457b3981b28b06e48a0de672620d6a03805be1f384e4ed3bf3a20f84ca50a76e: Status 404 returned error can't find the container with id 457b3981b28b06e48a0de672620d6a03805be1f384e4ed3bf3a20f84ca50a76e Jul 07 17:41:10.812567 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:10.812509 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" event={"ID":"55437412-4864-499e-82c2-a854b002bb47","Type":"ContainerStarted","Data":"d5987b26e658a54acebc813041d19ea2459f548de6affef263f803d405005270"} Jul 07 17:41:10.814812 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:10.814761 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8xtch" event={"ID":"f6702dde-6263-479f-933b-dcd73e1ed5ed","Type":"ContainerStarted","Data":"5c6273d5d54ab959a2b94ee159e20ad98396d7b11d76301fc70f0ae4784b3b92"} Jul 07 17:41:10.814989 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:10.814849 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8xtch" event={"ID":"f6702dde-6263-479f-933b-dcd73e1ed5ed","Type":"ContainerStarted","Data":"457b3981b28b06e48a0de672620d6a03805be1f384e4ed3bf3a20f84ca50a76e"} Jul 07 17:41:10.817456 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:10.817418 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" event={"ID":"b473a0e9-cbb7-404f-9f76-94e3bc7394ca","Type":"ContainerStarted","Data":"63e234d0b3508703c55387db20594cffec01c9e7e9411e930a5929400fdf730c"} Jul 07 17:41:10.820220 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:10.820187 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" event={"ID":"055ae3eb-05b3-4509-9008-d62baf858fb3","Type":"ContainerStarted","Data":"ebbde7e25007c185f2b8cb650f1bb0f10649e017f677dc62ab7fab04a085ff42"} Jul 07 17:41:10.821999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:10.821931 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-s9wsz" event={"ID":"344f0c5c-f855-4e9d-acf3-705194c9ade4","Type":"ContainerStarted","Data":"c618985ef182397ca0a6170b6f42b7f1d962610c3e40de4a77bc02c71b45d998"} Jul 07 17:41:10.823642 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:10.823610 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" event={"ID":"d707cdcf-3720-438d-a034-cf5dd54f2a47","Type":"ContainerStarted","Data":"805b8c90481b6e16309b4bb340cbf4692c9aa1ce010e338431fe613f1d64862d"} Jul 07 17:41:10.834154 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:10.833232 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8xtch" podStartSLOduration=1.833209331 podStartE2EDuration="1.833209331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:41:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:41:10.832424162 +0000 UTC m=+74.871751068" watchObservedRunningTime="2026-07-07 17:41:10.833209331 +0000 UTC m=+74.872536218" Jul 07 17:41:10.906975 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:10.906384 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:10.906975 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:10.906529 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 17:41:10.906975 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:10.906597 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls podName:2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:12.906574785 +0000 UTC m=+76.945901693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-jzjvx" (UID: "2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4") : secret "cluster-monitoring-operator-tls" not found Jul 07 17:41:11.008080 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:11.007597 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:11.008080 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:11.007723 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-k9wh9\" (UID: \"3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" Jul 07 17:41:11.008080 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:11.007756 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:11.008080 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:11.007798 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle podName:57a355c0-7af9-4412-b621-771fae3a122c nodeName:}" failed. No retries permitted until 2026-07-07 17:41:13.007774966 +0000 UTC m=+77.047101863 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle") pod "router-default-5c5c789b7f-26cs9" (UID: "57a355c0-7af9-4412-b621-771fae3a122c") : configmap references non-existent config key: service-ca.crt Jul 07 17:41:11.008080 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:11.007883 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 17:41:11.008080 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:11.007933 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs podName:57a355c0-7af9-4412-b621-771fae3a122c nodeName:}" failed. No retries permitted until 2026-07-07 17:41:13.007917592 +0000 UTC m=+77.047244494 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs") pod "router-default-5c5c789b7f-26cs9" (UID: "57a355c0-7af9-4412-b621-771fae3a122c") : secret "router-metrics-certs-default" not found Jul 07 17:41:11.008080 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:11.008000 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:41:11.008080 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:11.008032 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls podName:3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:13.00802203 +0000 UTC m=+77.047348914 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-k9wh9" (UID: "3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32") : secret "samples-operator-tls" not found Jul 07 17:41:12.833399 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:12.833357 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-s9wsz" event={"ID":"344f0c5c-f855-4e9d-acf3-705194c9ade4","Type":"ContainerStarted","Data":"2be8108c1165642ff4809953248283d312185fb2d7326d387a5a088fee1d0a1c"} Jul 07 17:41:12.851286 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:12.851223 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-s9wsz" podStartSLOduration=1.912880301 podStartE2EDuration="3.851202298s" totalImagesPullingTime="1.938321997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:41:12.850127652 +0000 UTC m=+76.889454554" watchObservedRunningTime="2026-07-07 17:41:12.851202298 +0000 UTC m=+76.890529199" Jul 07 17:41:12.928599 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:12.928026 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:12.928599 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:12.928173 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 17:41:12.928599 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:12.928249 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls podName:2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:16.928227868 +0000 UTC m=+80.967554751 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-jzjvx" (UID: "2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4") : secret "cluster-monitoring-operator-tls" not found Jul 07 17:41:13.028995 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:13.028948 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:13.029206 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:13.029101 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-k9wh9\" (UID: \"3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" Jul 07 17:41:13.029206 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:13.029144 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:13.029206 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:13.029168 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle podName:57a355c0-7af9-4412-b621-771fae3a122c nodeName:}" failed. No retries permitted until 2026-07-07 17:41:17.02914547 +0000 UTC m=+81.068472353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle") pod "router-default-5c5c789b7f-26cs9" (UID: "57a355c0-7af9-4412-b621-771fae3a122c") : configmap references non-existent config key: service-ca.crt Jul 07 17:41:13.029364 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:13.029232 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 17:41:13.029364 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:13.029264 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:41:13.029364 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:13.029283 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs podName:57a355c0-7af9-4412-b621-771fae3a122c nodeName:}" failed. No retries permitted until 2026-07-07 17:41:17.029266663 +0000 UTC m=+81.068593559 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs") pod "router-default-5c5c789b7f-26cs9" (UID: "57a355c0-7af9-4412-b621-771fae3a122c") : secret "router-metrics-certs-default" not found Jul 07 17:41:13.029364 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:13.029320 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls podName:3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:17.029304424 +0000 UTC m=+81.068631320 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-k9wh9" (UID: "3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32") : secret "samples-operator-tls" not found Jul 07 17:41:14.843205 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:14.843160 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" event={"ID":"055ae3eb-05b3-4509-9008-d62baf858fb3","Type":"ContainerStarted","Data":"fe532d08ec833c3243726ce279c11c50dcf236d639534f1a3077be7d708a7231"} Jul 07 17:41:14.844813 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:14.844779 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" event={"ID":"d707cdcf-3720-438d-a034-cf5dd54f2a47","Type":"ContainerStarted","Data":"e806035eeb9752c5030e1bed4de9819a05930af57202993328969f3cd6eb9ab8"} Jul 07 17:41:14.846446 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:14.846416 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" event={"ID":"55437412-4864-499e-82c2-a854b002bb47","Type":"ContainerStarted","Data":"0a3a53149f6acd8fc2c9bd37391fe0ed0ca3ea96f0fbdecfda64208eb1ff5c4e"} Jul 07 17:41:14.848543 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:14.848514 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/0.log" Jul 07 17:41:14.848693 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:14.848560 2577 generic.go:356] "Generic (PLEG): container finished" podID="b473a0e9-cbb7-404f-9f76-94e3bc7394ca" containerID="2d5337b04ea3dc89ed5b37c44f328b257d20a2cb5c9681885533dc67da346ed9" exitCode=255 Jul 07 17:41:14.848693 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:14.848589 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" event={"ID":"b473a0e9-cbb7-404f-9f76-94e3bc7394ca","Type":"ContainerDied","Data":"2d5337b04ea3dc89ed5b37c44f328b257d20a2cb5c9681885533dc67da346ed9"} Jul 07 17:41:14.848865 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:14.848847 2577 scope.go:117] "RemoveContainer" containerID="2d5337b04ea3dc89ed5b37c44f328b257d20a2cb5c9681885533dc67da346ed9" Jul 07 17:41:14.866933 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:14.866871 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" podStartSLOduration=1.4305527009999999 podStartE2EDuration="5.866845881s" totalImagesPullingTime="4.43629318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:41:14.866721256 +0000 UTC m=+78.906048161" watchObservedRunningTime="2026-07-07 17:41:14.866845881 +0000 UTC m=+78.906172785" Jul 07 17:41:14.890954 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:14.890894 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" podStartSLOduration=1.540787592 podStartE2EDuration="5.890873023s" totalImagesPullingTime="4.350085431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:41:14.889296188 +0000 UTC m=+78.928623091" watchObservedRunningTime="2026-07-07 17:41:14.890873023 +0000 UTC m=+78.930199957" Jul 07 17:41:14.957511 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:14.957442 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" podStartSLOduration=1.488788285 podStartE2EDuration="5.957420884s" totalImagesPullingTime="4.468632599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:41:14.956291836 +0000 UTC m=+78.995618739" watchObservedRunningTime="2026-07-07 17:41:14.957420884 +0000 UTC m=+78.996747788" Jul 07 17:41:15.854564 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:15.854529 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/1.log" Jul 07 17:41:15.855077 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:15.855031 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/0.log" Jul 07 17:41:15.855218 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:15.855097 2577 generic.go:356] "Generic (PLEG): container finished" podID="b473a0e9-cbb7-404f-9f76-94e3bc7394ca" containerID="af0e357abefbfb6e2d7a3d4dbeda41360b6a1c5ac3f266f3872d513cad142904" exitCode=255 Jul 07 17:41:15.855281 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:15.855218 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" event={"ID":"b473a0e9-cbb7-404f-9f76-94e3bc7394ca","Type":"ContainerDied","Data":"af0e357abefbfb6e2d7a3d4dbeda41360b6a1c5ac3f266f3872d513cad142904"} Jul 07 17:41:15.855335 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:15.855321 2577 scope.go:117] "RemoveContainer" containerID="2d5337b04ea3dc89ed5b37c44f328b257d20a2cb5c9681885533dc67da346ed9" Jul 07 17:41:15.855488 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:15.855441 2577 scope.go:117] "RemoveContainer" containerID="af0e357abefbfb6e2d7a3d4dbeda41360b6a1c5ac3f266f3872d513cad142904" Jul 07 17:41:15.855651 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:15.855628 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-5brrn_openshift-console-operator(b473a0e9-cbb7-404f-9f76-94e3bc7394ca)\"" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" podUID="b473a0e9-cbb7-404f-9f76-94e3bc7394ca" Jul 07 17:41:16.064736 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.064682 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7j42m"] Jul 07 17:41:16.069305 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.069278 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.081688 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.081651 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 17:41:16.081688 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.081666 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 17:41:16.081905 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.081763 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hzvmm\"" Jul 07 17:41:16.100458 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.100422 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7j42m"] Jul 07 17:41:16.157426 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.157329 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tttr\" (UniqueName: \"kubernetes.io/projected/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-api-access-6tttr\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.157426 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.157386 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-crio-socket\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.157613 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.157466 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.157613 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.157557 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.157613 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.157579 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-data-volume\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.258715 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.258671 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.258715 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.258718 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-data-volume\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.258980 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.258769 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tttr\" (UniqueName: \"kubernetes.io/projected/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-api-access-6tttr\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.258980 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.258806 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-crio-socket\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.258980 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:16.258838 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:41:16.258980 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.258857 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.258980 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:16.258926 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls podName:5313c0af-946a-4e13-91f7-e2f1a5d05bb2 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:16.758905794 +0000 UTC m=+80.798232693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7j42m" (UID: "5313c0af-946a-4e13-91f7-e2f1a5d05bb2") : secret "insights-runtime-extractor-tls" not found Jul 07 17:41:16.258980 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.258959 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-crio-socket\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.258980 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:16.258979 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 17:41:16.259317 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:16.259081 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm podName:5313c0af-946a-4e13-91f7-e2f1a5d05bb2 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:16.759033568 +0000 UTC m=+80.798360457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7j42m" (UID: "5313c0af-946a-4e13-91f7-e2f1a5d05bb2") : configmap "kube-rbac-proxy" not found Jul 07 17:41:16.259317 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.259153 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-data-volume\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.270579 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.270552 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tttr\" (UniqueName: \"kubernetes.io/projected/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-api-access-6tttr\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.763753 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.763707 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.763932 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.763833 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:16.763932 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:16.763836 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 17:41:16.763932 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:16.763913 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:41:16.764031 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:16.763960 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm podName:5313c0af-946a-4e13-91f7-e2f1a5d05bb2 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:17.763941218 +0000 UTC m=+81.803268103 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7j42m" (UID: "5313c0af-946a-4e13-91f7-e2f1a5d05bb2") : configmap "kube-rbac-proxy" not found Jul 07 17:41:16.764031 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:16.763975 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls podName:5313c0af-946a-4e13-91f7-e2f1a5d05bb2 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:17.763968707 +0000 UTC m=+81.803295587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7j42m" (UID: "5313c0af-946a-4e13-91f7-e2f1a5d05bb2") : secret "insights-runtime-extractor-tls" not found Jul 07 17:41:16.864931 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.864904 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/1.log" Jul 07 17:41:16.865311 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.865259 2577 scope.go:117] "RemoveContainer" containerID="af0e357abefbfb6e2d7a3d4dbeda41360b6a1c5ac3f266f3872d513cad142904" Jul 07 17:41:16.865427 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:16.865406 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-5brrn_openshift-console-operator(b473a0e9-cbb7-404f-9f76-94e3bc7394ca)\"" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" podUID="b473a0e9-cbb7-404f-9f76-94e3bc7394ca" Jul 07 17:41:16.965447 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:16.965301 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:16.965447 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:16.965357 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 17:41:16.965447 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:16.965425 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls podName:2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:24.965411086 +0000 UTC m=+89.004737967 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-jzjvx" (UID: "2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4") : secret "cluster-monitoring-operator-tls" not found Jul 07 17:41:17.066751 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:17.066643 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:17.066942 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:17.066754 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-k9wh9\" (UID: \"3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" Jul 07 17:41:17.066942 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:17.066775 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:17.066942 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:17.066855 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle podName:57a355c0-7af9-4412-b621-771fae3a122c nodeName:}" failed. No retries permitted until 2026-07-07 17:41:25.066829907 +0000 UTC m=+89.106156788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle") pod "router-default-5c5c789b7f-26cs9" (UID: "57a355c0-7af9-4412-b621-771fae3a122c") : configmap references non-existent config key: service-ca.crt Jul 07 17:41:17.066942 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:17.066901 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 17:41:17.067196 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:17.066951 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls podName:3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:25.066938224 +0000 UTC m=+89.106265105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-k9wh9" (UID: "3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32") : secret "samples-operator-tls" not found Jul 07 17:41:17.067196 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:17.066902 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 17:41:17.067196 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:17.066982 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs podName:57a355c0-7af9-4412-b621-771fae3a122c nodeName:}" failed. No retries permitted until 2026-07-07 17:41:25.06697559 +0000 UTC m=+89.106302471 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs") pod "router-default-5c5c789b7f-26cs9" (UID: "57a355c0-7af9-4412-b621-771fae3a122c") : secret "router-metrics-certs-default" not found Jul 07 17:41:17.773573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:17.773509 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:17.773763 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:17.773636 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:17.773763 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:17.773689 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:41:17.773763 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:17.773734 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 17:41:17.773763 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:17.773760 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls podName:5313c0af-946a-4e13-91f7-e2f1a5d05bb2 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:19.773745055 +0000 UTC m=+83.813071936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7j42m" (UID: "5313c0af-946a-4e13-91f7-e2f1a5d05bb2") : secret "insights-runtime-extractor-tls" not found Jul 07 17:41:17.773901 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:17.773791 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm podName:5313c0af-946a-4e13-91f7-e2f1a5d05bb2 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:19.773768608 +0000 UTC m=+83.813095493 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7j42m" (UID: "5313c0af-946a-4e13-91f7-e2f1a5d05bb2") : configmap "kube-rbac-proxy" not found Jul 07 17:41:18.441952 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:18.441923 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nck8x_d1d8d149-d9ff-4b3c-851f-ec9de53109b6/dns-node-resolver/0.log" Jul 07 17:41:19.638354 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:19.638329 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t5q9p_4db38d2b-e752-4fdf-81ed-d134f7831397/node-ca/0.log" Jul 07 17:41:19.792729 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:19.792692 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:19.792925 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:19.792796 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:19.792925 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:19.792847 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 17:41:19.792925 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:19.792909 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm podName:5313c0af-946a-4e13-91f7-e2f1a5d05bb2 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:23.792895285 +0000 UTC m=+87.832222182 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7j42m" (UID: "5313c0af-946a-4e13-91f7-e2f1a5d05bb2") : configmap "kube-rbac-proxy" not found Jul 07 17:41:19.793075 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:19.792957 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:41:19.793075 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:19.793029 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls podName:5313c0af-946a-4e13-91f7-e2f1a5d05bb2 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:23.793016043 +0000 UTC m=+87.832342926 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7j42m" (UID: "5313c0af-946a-4e13-91f7-e2f1a5d05bb2") : secret "insights-runtime-extractor-tls" not found Jul 07 17:41:19.856766 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:19.856721 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:19.856929 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:19.856788 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:19.857186 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:19.857173 2577 scope.go:117] "RemoveContainer" containerID="af0e357abefbfb6e2d7a3d4dbeda41360b6a1c5ac3f266f3872d513cad142904" Jul 07 17:41:19.857345 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:19.857327 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-5brrn_openshift-console-operator(b473a0e9-cbb7-404f-9f76-94e3bc7394ca)\"" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" podUID="b473a0e9-cbb7-404f-9f76-94e3bc7394ca" Jul 07 17:41:20.240461 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:20.240429 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x5q45_258e1cc3-989f-4b11-9152-31b2aeb13bb9/serve-healthcheck-canary/0.log" Jul 07 17:41:20.841800 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:20.841765 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-dwjq4_55437412-4864-499e-82c2-a854b002bb47/kube-storage-version-migrator-operator/0.log" Jul 07 17:41:23.829298 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:23.829255 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:23.829773 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:23.829346 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:23.829773 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:23.829427 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 17:41:23.829773 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:23.829428 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:41:23.829773 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:23.829491 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm podName:5313c0af-946a-4e13-91f7-e2f1a5d05bb2 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:31.829477624 +0000 UTC m=+95.868804505 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7j42m" (UID: "5313c0af-946a-4e13-91f7-e2f1a5d05bb2") : configmap "kube-rbac-proxy" not found Jul 07 17:41:23.829773 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:23.829504 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls podName:5313c0af-946a-4e13-91f7-e2f1a5d05bb2 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:31.829498629 +0000 UTC m=+95.868825509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7j42m" (UID: "5313c0af-946a-4e13-91f7-e2f1a5d05bb2") : secret "insights-runtime-extractor-tls" not found Jul 07 17:41:25.037836 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:25.037782 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:25.038280 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:25.037954 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 17:41:25.038280 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:25.038019 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls podName:2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:41.038004087 +0000 UTC m=+105.077330968 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-jzjvx" (UID: "2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4") : secret "cluster-monitoring-operator-tls" not found Jul 07 17:41:25.139112 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:25.139073 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-k9wh9\" (UID: \"3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" Jul 07 17:41:25.139112 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:25.139115 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:25.139375 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:25.139192 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:25.139375 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:25.139281 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 17:41:25.139375 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:25.139336 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle podName:57a355c0-7af9-4412-b621-771fae3a122c nodeName:}" failed. No retries permitted until 2026-07-07 17:41:41.139321277 +0000 UTC m=+105.178648158 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle") pod "router-default-5c5c789b7f-26cs9" (UID: "57a355c0-7af9-4412-b621-771fae3a122c") : configmap references non-existent config key: service-ca.crt Jul 07 17:41:25.139375 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:25.139352 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs podName:57a355c0-7af9-4412-b621-771fae3a122c nodeName:}" failed. No retries permitted until 2026-07-07 17:41:41.139345173 +0000 UTC m=+105.178672054 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs") pod "router-default-5c5c789b7f-26cs9" (UID: "57a355c0-7af9-4412-b621-771fae3a122c") : secret "router-metrics-certs-default" not found Jul 07 17:41:25.142014 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:25.141975 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-k9wh9\" (UID: \"3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" Jul 07 17:41:25.240286 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:25.240250 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" Jul 07 17:41:25.407320 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:25.407284 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9"] Jul 07 17:41:25.893939 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:25.893899 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" event={"ID":"3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32","Type":"ContainerStarted","Data":"ebe469f3669f54b26607289f12d4a36b599c9afa5167dc14a1820e442567e6c8"} Jul 07 17:41:27.902701 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:27.902602 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" event={"ID":"3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32","Type":"ContainerStarted","Data":"1cd68f91422dbaaf4ed85cb908b0e6873d1e04f5372f8bd9d0a60bb18b2422d9"} Jul 07 17:41:27.902701 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:27.902663 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" event={"ID":"3f7ddbb7-2bc4-48bd-9a6b-98a233be0b32","Type":"ContainerStarted","Data":"d9b3c7624fee4dd0dce10849fb92c7aa0cb59a4a5cfd7d8a8576171b27140646"} Jul 07 17:41:27.920253 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:27.920192 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-k9wh9" podStartSLOduration=16.908969388 podStartE2EDuration="18.920176032s" totalImagesPullingTime="2.011206644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:41:27.920013829 +0000 UTC m=+91.959340732" watchObservedRunningTime="2026-07-07 17:41:27.920176032 +0000 UTC m=+91.959502913" Jul 07 17:41:31.904194 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:31.904148 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:31.904623 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:31.904239 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:31.904623 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:31.904311 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 17:41:31.904623 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:31.904387 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls podName:5313c0af-946a-4e13-91f7-e2f1a5d05bb2 nodeName:}" failed. No retries permitted until 2026-07-07 17:41:47.904371192 +0000 UTC m=+111.943698072 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7j42m" (UID: "5313c0af-946a-4e13-91f7-e2f1a5d05bb2") : secret "insights-runtime-extractor-tls" not found Jul 07 17:41:31.904737 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:31.904705 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:35.511322 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:35.511287 2577 scope.go:117] "RemoveContainer" containerID="af0e357abefbfb6e2d7a3d4dbeda41360b6a1c5ac3f266f3872d513cad142904" Jul 07 17:41:35.927917 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:35.927831 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 17:41:35.928242 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:35.928225 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/1.log" Jul 07 17:41:35.928361 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:35.928263 2577 generic.go:356] "Generic (PLEG): container finished" podID="b473a0e9-cbb7-404f-9f76-94e3bc7394ca" containerID="4b848b614abb4f156182bba3fe142656eff19c1a35bcb56bcb62537a73f328dd" exitCode=255 Jul 07 17:41:35.928361 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:35.928330 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" event={"ID":"b473a0e9-cbb7-404f-9f76-94e3bc7394ca","Type":"ContainerDied","Data":"4b848b614abb4f156182bba3fe142656eff19c1a35bcb56bcb62537a73f328dd"} Jul 07 17:41:35.928464 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:35.928392 2577 scope.go:117] "RemoveContainer" containerID="af0e357abefbfb6e2d7a3d4dbeda41360b6a1c5ac3f266f3872d513cad142904" Jul 07 17:41:35.928728 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:35.928707 2577 scope.go:117] "RemoveContainer" containerID="4b848b614abb4f156182bba3fe142656eff19c1a35bcb56bcb62537a73f328dd" Jul 07 17:41:35.928933 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:35.928909 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-5brrn_openshift-console-operator(b473a0e9-cbb7-404f-9f76-94e3bc7394ca)\"" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" podUID="b473a0e9-cbb7-404f-9f76-94e3bc7394ca" Jul 07 17:41:36.932999 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:36.932969 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 17:41:37.253442 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.253354 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:41:37.253577 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.253453 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:41:37.255986 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.255942 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71-metrics-tls\") pod \"dns-default-b2xkh\" (UID: \"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71\") " pod="openshift-dns/dns-default-b2xkh" Jul 07 17:41:37.256157 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.256013 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls\") pod \"image-registry-675994669b-7mkgg\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:41:37.523619 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.523530 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vx5vk\"" Jul 07 17:41:37.530513 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.530489 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:41:37.542221 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.542192 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6c56v\"" Jul 07 17:41:37.548557 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.548306 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b2xkh" Jul 07 17:41:37.727588 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.727554 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-675994669b-7mkgg"] Jul 07 17:41:37.730336 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.730305 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b2xkh"] Jul 07 17:41:37.730945 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:41:37.730919 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89619451_c733_4743_90b3_ec2cffa338fc.slice/crio-dae6a14f5bd936b6125811bc092d495b9b1929fae7340e10f02b0713dd6d4038 WatchSource:0}: Error finding container dae6a14f5bd936b6125811bc092d495b9b1929fae7340e10f02b0713dd6d4038: Status 404 returned error can't find the container with id dae6a14f5bd936b6125811bc092d495b9b1929fae7340e10f02b0713dd6d4038 Jul 07 17:41:37.733512 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:41:37.733491 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a1b3e2f_62eb_4dc7_8995_ca7ab8635f71.slice/crio-7a2a3bec8f03cc5500976d14de33567f87175651033e0e0196dfd982fe2dbee2 WatchSource:0}: Error finding container 7a2a3bec8f03cc5500976d14de33567f87175651033e0e0196dfd982fe2dbee2: Status 404 returned error can't find the container with id 7a2a3bec8f03cc5500976d14de33567f87175651033e0e0196dfd982fe2dbee2 Jul 07 17:41:37.938438 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.938397 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b2xkh" event={"ID":"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71","Type":"ContainerStarted","Data":"7a2a3bec8f03cc5500976d14de33567f87175651033e0e0196dfd982fe2dbee2"} Jul 07 17:41:37.940074 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.940023 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-675994669b-7mkgg" event={"ID":"89619451-c733-4743-90b3-ec2cffa338fc","Type":"ContainerStarted","Data":"34eb29a02eb6a21d4be299741265a1086fea89184d46f41ad360af352d0542fd"} Jul 07 17:41:37.940233 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.940098 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-675994669b-7mkgg" event={"ID":"89619451-c733-4743-90b3-ec2cffa338fc","Type":"ContainerStarted","Data":"dae6a14f5bd936b6125811bc092d495b9b1929fae7340e10f02b0713dd6d4038"} Jul 07 17:41:37.940233 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.940167 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:41:37.994781 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:37.994708 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-675994669b-7mkgg" podStartSLOduration=100.99465856 podStartE2EDuration="1m40.99465856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:39:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:41:37.99457695 +0000 UTC m=+102.033903853" watchObservedRunningTime="2026-07-07 17:41:37.99465856 +0000 UTC m=+102.033985465" Jul 07 17:41:39.809419 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:39.809385 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qtftl" Jul 07 17:41:39.857057 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:39.857004 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:39.857237 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:39.857093 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:41:39.857405 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:39.857393 2577 scope.go:117] "RemoveContainer" containerID="4b848b614abb4f156182bba3fe142656eff19c1a35bcb56bcb62537a73f328dd" Jul 07 17:41:39.857563 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:39.857545 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-5brrn_openshift-console-operator(b473a0e9-cbb7-404f-9f76-94e3bc7394ca)\"" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" podUID="b473a0e9-cbb7-404f-9f76-94e3bc7394ca" Jul 07 17:41:39.949281 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:39.949242 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b2xkh" event={"ID":"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71","Type":"ContainerStarted","Data":"3a98de8f144c2bb266f3216e46dbcbd128b8c993dac22fd3ff18a0cfea813f45"} Jul 07 17:41:39.949480 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:39.949299 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b2xkh" event={"ID":"8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71","Type":"ContainerStarted","Data":"bc2394db416fc842a7e54f81aff92f910f899eaa4b1710f9a87c9d935149cfa7"} Jul 07 17:41:39.949480 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:39.949430 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b2xkh" Jul 07 17:41:39.980767 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:39.980697 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b2xkh" podStartSLOduration=65.220896752 podStartE2EDuration="1m6.980679139s" totalImagesPullingTime="1.759782387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:41:39.980284914 +0000 UTC m=+104.019611818" watchObservedRunningTime="2026-07-07 17:41:39.980679139 +0000 UTC m=+104.020006076" Jul 07 17:41:41.088623 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:41.088575 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:41.091180 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:41.091157 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-jzjvx\" (UID: \"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:41.189634 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:41.189591 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:41.189805 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:41.189696 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:41.190478 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:41.190437 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57a355c0-7af9-4412-b621-771fae3a122c-service-ca-bundle\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:41.192373 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:41.192348 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57a355c0-7af9-4412-b621-771fae3a122c-metrics-certs\") pod \"router-default-5c5c789b7f-26cs9\" (UID: \"57a355c0-7af9-4412-b621-771fae3a122c\") " pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:41.328062 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:41.328009 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" Jul 07 17:41:41.468221 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:41.468183 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:41.513398 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:41:41.513363 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d8f2aa1_a7fe_4b51_8e61_5eb356bd27e4.slice/crio-abea00f2627f2ecd1be1b2c7bb80377df19ffa511f12b4f45afbe638ab6a097e WatchSource:0}: Error finding container abea00f2627f2ecd1be1b2c7bb80377df19ffa511f12b4f45afbe638ab6a097e: Status 404 returned error can't find the container with id abea00f2627f2ecd1be1b2c7bb80377df19ffa511f12b4f45afbe638ab6a097e Jul 07 17:41:41.531418 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:41.531375 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx"] Jul 07 17:41:41.653556 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:41.653461 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5c5c789b7f-26cs9"] Jul 07 17:41:41.660194 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:41:41.660165 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57a355c0_7af9_4412_b621_771fae3a122c.slice/crio-771ab6e554347475f342caf23821375f8002de30f73c81682e969940ec62067a WatchSource:0}: Error finding container 771ab6e554347475f342caf23821375f8002de30f73c81682e969940ec62067a: Status 404 returned error can't find the container with id 771ab6e554347475f342caf23821375f8002de30f73c81682e969940ec62067a Jul 07 17:41:41.959034 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:41.958997 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" event={"ID":"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4","Type":"ContainerStarted","Data":"abea00f2627f2ecd1be1b2c7bb80377df19ffa511f12b4f45afbe638ab6a097e"} Jul 07 17:41:41.960531 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:41.960504 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5c5c789b7f-26cs9" event={"ID":"57a355c0-7af9-4412-b621-771fae3a122c","Type":"ContainerStarted","Data":"31b3140bf56fff8f6bee41f0d03e509f9e50afa140fb866a1d404f17360df76a"} Jul 07 17:41:41.960674 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:41.960547 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5c5c789b7f-26cs9" event={"ID":"57a355c0-7af9-4412-b621-771fae3a122c","Type":"ContainerStarted","Data":"771ab6e554347475f342caf23821375f8002de30f73c81682e969940ec62067a"} Jul 07 17:41:41.991518 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:41.991438 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5c5c789b7f-26cs9" podStartSLOduration=32.991415269 podStartE2EDuration="32.991415269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:41:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:41:41.991373388 +0000 UTC m=+106.030700292" watchObservedRunningTime="2026-07-07 17:41:41.991415269 +0000 UTC m=+106.030742172" Jul 07 17:41:42.469016 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:42.468976 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:42.471722 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:42.471692 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:42.965339 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:42.965298 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:42.966806 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:42.966776 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5c5c789b7f-26cs9" Jul 07 17:41:43.726873 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.726834 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk"] Jul 07 17:41:43.767013 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.766978 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk"] Jul 07 17:41:43.767223 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.767135 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" Jul 07 17:41:43.771888 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.771856 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 17:41:43.772770 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.772752 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 07 17:41:43.772863 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.772844 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 17:41:43.773811 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.773791 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 17:41:43.812941 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.812902 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79351d81-5f50-4f49-b563-89e8bdb23890-tmp\") pod \"klusterlet-addon-workmgr-7b48497b46-ctzxk\" (UID: \"79351d81-5f50-4f49-b563-89e8bdb23890\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" Jul 07 17:41:43.813129 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.812960 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/79351d81-5f50-4f49-b563-89e8bdb23890-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b48497b46-ctzxk\" (UID: \"79351d81-5f50-4f49-b563-89e8bdb23890\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" Jul 07 17:41:43.813129 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.813002 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbfc2\" (UniqueName: \"kubernetes.io/projected/79351d81-5f50-4f49-b563-89e8bdb23890-kube-api-access-nbfc2\") pod \"klusterlet-addon-workmgr-7b48497b46-ctzxk\" (UID: \"79351d81-5f50-4f49-b563-89e8bdb23890\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" Jul 07 17:41:43.913806 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.913763 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79351d81-5f50-4f49-b563-89e8bdb23890-tmp\") pod \"klusterlet-addon-workmgr-7b48497b46-ctzxk\" (UID: \"79351d81-5f50-4f49-b563-89e8bdb23890\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" Jul 07 17:41:43.913998 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.913832 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/79351d81-5f50-4f49-b563-89e8bdb23890-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b48497b46-ctzxk\" (UID: \"79351d81-5f50-4f49-b563-89e8bdb23890\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" Jul 07 17:41:43.913998 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.913857 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbfc2\" (UniqueName: \"kubernetes.io/projected/79351d81-5f50-4f49-b563-89e8bdb23890-kube-api-access-nbfc2\") pod \"klusterlet-addon-workmgr-7b48497b46-ctzxk\" (UID: \"79351d81-5f50-4f49-b563-89e8bdb23890\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" Jul 07 17:41:43.914276 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.914254 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79351d81-5f50-4f49-b563-89e8bdb23890-tmp\") pod \"klusterlet-addon-workmgr-7b48497b46-ctzxk\" (UID: \"79351d81-5f50-4f49-b563-89e8bdb23890\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" Jul 07 17:41:43.916666 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.916638 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/79351d81-5f50-4f49-b563-89e8bdb23890-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b48497b46-ctzxk\" (UID: \"79351d81-5f50-4f49-b563-89e8bdb23890\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" Jul 07 17:41:43.942907 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:43.942812 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbfc2\" (UniqueName: \"kubernetes.io/projected/79351d81-5f50-4f49-b563-89e8bdb23890-kube-api-access-nbfc2\") pod \"klusterlet-addon-workmgr-7b48497b46-ctzxk\" (UID: \"79351d81-5f50-4f49-b563-89e8bdb23890\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" Jul 07 17:41:44.077699 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:44.077652 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" Jul 07 17:41:44.368540 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:44.368514 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk"] Jul 07 17:41:44.370880 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:41:44.370836 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79351d81_5f50_4f49_b563_89e8bdb23890.slice/crio-09774e284a83d4929f5677d4818feaa0001b9791ccb59932aa6f46a606abab1e WatchSource:0}: Error finding container 09774e284a83d4929f5677d4818feaa0001b9791ccb59932aa6f46a606abab1e: Status 404 returned error can't find the container with id 09774e284a83d4929f5677d4818feaa0001b9791ccb59932aa6f46a606abab1e Jul 07 17:41:44.973052 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:44.973015 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" event={"ID":"2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4","Type":"ContainerStarted","Data":"a1a2e503423bc477b9c0a4b6b48ba5293a3538388c369947f73141db34e44838"} Jul 07 17:41:44.974214 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:44.974186 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" event={"ID":"79351d81-5f50-4f49-b563-89e8bdb23890","Type":"ContainerStarted","Data":"09774e284a83d4929f5677d4818feaa0001b9791ccb59932aa6f46a606abab1e"} Jul 07 17:41:45.049977 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:45.049920 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-jzjvx" podStartSLOduration=33.35847137 podStartE2EDuration="36.049901653s" totalImagesPullingTime="2.691430283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:41:45.049773795 +0000 UTC m=+109.089100708" watchObservedRunningTime="2026-07-07 17:41:45.049901653 +0000 UTC m=+109.089228556" Jul 07 17:41:47.950443 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:47.950398 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:47.952872 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:47.952845 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5313c0af-946a-4e13-91f7-e2f1a5d05bb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7j42m\" (UID: \"5313c0af-946a-4e13-91f7-e2f1a5d05bb2\") " pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:48.178695 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:48.178633 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7j42m" Jul 07 17:41:48.318911 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:48.318878 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7j42m"] Jul 07 17:41:48.322523 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:41:48.322493 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5313c0af_946a_4e13_91f7_e2f1a5d05bb2.slice/crio-5dbc159b6375e6134ebf4fe628f4f4108e7dc132e101e1ddce2b241405a62ef4 WatchSource:0}: Error finding container 5dbc159b6375e6134ebf4fe628f4f4108e7dc132e101e1ddce2b241405a62ef4: Status 404 returned error can't find the container with id 5dbc159b6375e6134ebf4fe628f4f4108e7dc132e101e1ddce2b241405a62ef4 Jul 07 17:41:48.991914 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:48.991867 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7j42m" event={"ID":"5313c0af-946a-4e13-91f7-e2f1a5d05bb2","Type":"ContainerStarted","Data":"5dbc159b6375e6134ebf4fe628f4f4108e7dc132e101e1ddce2b241405a62ef4"} Jul 07 17:41:49.956268 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:49.956233 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b2xkh" Jul 07 17:41:53.009916 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:53.009878 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7j42m" event={"ID":"5313c0af-946a-4e13-91f7-e2f1a5d05bb2","Type":"ContainerStarted","Data":"2ef037db0c650ff1b26a3f2fdd6028133a3e5c313090206e53e3262159f81abb"} Jul 07 17:41:53.010340 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:53.009968 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7j42m" event={"ID":"5313c0af-946a-4e13-91f7-e2f1a5d05bb2","Type":"ContainerStarted","Data":"34ea540e9d9468c0e6097cdd91b3007590f05ee880a5b72e3e1f08771856ea00"} Jul 07 17:41:53.510794 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:53.510759 2577 scope.go:117] "RemoveContainer" containerID="4b848b614abb4f156182bba3fe142656eff19c1a35bcb56bcb62537a73f328dd" Jul 07 17:41:53.510982 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:41:53.510960 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-5brrn_openshift-console-operator(b473a0e9-cbb7-404f-9f76-94e3bc7394ca)\"" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" podUID="b473a0e9-cbb7-404f-9f76-94e3bc7394ca" Jul 07 17:41:56.027487 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:56.027449 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7j42m" event={"ID":"5313c0af-946a-4e13-91f7-e2f1a5d05bb2","Type":"ContainerStarted","Data":"1d029abb2753797a7251f2810396b29e7b0efa43cf227d88a1eb38a096e13f84"} Jul 07 17:41:56.060904 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:56.060828 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7j42m" podStartSLOduration=33.137757707 podStartE2EDuration="40.060811437s" totalImagesPullingTime="6.92305373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:41:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:41:56.059467049 +0000 UTC m=+120.098793948" watchObservedRunningTime="2026-07-07 17:41:56.060811437 +0000 UTC m=+120.100138397" Jul 07 17:41:58.948429 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:58.948395 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:41:59.861247 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.861210 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tfzd2"] Jul 07 17:41:59.865924 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.865895 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:41:59.868670 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.868642 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 17:41:59.868948 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.868931 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 17:41:59.869178 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.869164 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 17:41:59.870630 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.870609 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 17:41:59.870926 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.870911 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l27bs\"" Jul 07 17:41:59.953529 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.953488 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/786621af-8860-4858-a4a8-f9bc5ed7c9fe-metrics-client-ca\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:41:59.953529 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.953533 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-wtmp\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:41:59.954008 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.953554 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-textfile\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:41:59.954008 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.953573 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-accelerators-collector-config\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:41:59.954008 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.953627 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/786621af-8860-4858-a4a8-f9bc5ed7c9fe-sys\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:41:59.954008 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.953666 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:41:59.954008 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.953692 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27d84\" (UniqueName: \"kubernetes.io/projected/786621af-8860-4858-a4a8-f9bc5ed7c9fe-kube-api-access-27d84\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:41:59.954008 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.953713 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-tls\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:41:59.954008 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:41:59.953817 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/786621af-8860-4858-a4a8-f9bc5ed7c9fe-root\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055113 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055072 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/786621af-8860-4858-a4a8-f9bc5ed7c9fe-metrics-client-ca\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055292 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055125 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-wtmp\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055292 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055149 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-textfile\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055292 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055167 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-accelerators-collector-config\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055292 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055197 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/786621af-8860-4858-a4a8-f9bc5ed7c9fe-sys\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055292 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055221 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055292 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055247 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27d84\" (UniqueName: \"kubernetes.io/projected/786621af-8860-4858-a4a8-f9bc5ed7c9fe-kube-api-access-27d84\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055292 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055271 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-tls\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055621 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055340 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-wtmp\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055621 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055345 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/786621af-8860-4858-a4a8-f9bc5ed7c9fe-root\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055621 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055375 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/786621af-8860-4858-a4a8-f9bc5ed7c9fe-root\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055621 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055443 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/786621af-8860-4858-a4a8-f9bc5ed7c9fe-sys\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055803 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055708 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/786621af-8860-4858-a4a8-f9bc5ed7c9fe-metrics-client-ca\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055803 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055719 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-textfile\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.055950 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.055922 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-accelerators-collector-config\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.059313 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.059279 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.059451 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.059327 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/786621af-8860-4858-a4a8-f9bc5ed7c9fe-node-exporter-tls\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.075951 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.075916 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27d84\" (UniqueName: \"kubernetes.io/projected/786621af-8860-4858-a4a8-f9bc5ed7c9fe-kube-api-access-27d84\") pod \"node-exporter-tfzd2\" (UID: \"786621af-8860-4858-a4a8-f9bc5ed7c9fe\") " pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.176663 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:00.176580 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tfzd2" Jul 07 17:42:00.189004 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:42:00.188960 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod786621af_8860_4858_a4a8_f9bc5ed7c9fe.slice/crio-8cfe4b0554f19bd0cb477e5b52062f8cec184291a4a52679a744bf3c760f821f WatchSource:0}: Error finding container 8cfe4b0554f19bd0cb477e5b52062f8cec184291a4a52679a744bf3c760f821f: Status 404 returned error can't find the container with id 8cfe4b0554f19bd0cb477e5b52062f8cec184291a4a52679a744bf3c760f821f Jul 07 17:42:01.049437 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:01.049401 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfzd2" event={"ID":"786621af-8860-4858-a4a8-f9bc5ed7c9fe","Type":"ContainerStarted","Data":"8cfe4b0554f19bd0cb477e5b52062f8cec184291a4a52679a744bf3c760f821f"} Jul 07 17:42:02.055186 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:02.055139 2577 generic.go:356] "Generic (PLEG): container finished" podID="786621af-8860-4858-a4a8-f9bc5ed7c9fe" containerID="56d3dff4a57dfdf736588d18d3803ff799969b0e0c89d0f3f9c26a0093414d0d" exitCode=0 Jul 07 17:42:02.055186 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:02.055187 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfzd2" event={"ID":"786621af-8860-4858-a4a8-f9bc5ed7c9fe","Type":"ContainerDied","Data":"56d3dff4a57dfdf736588d18d3803ff799969b0e0c89d0f3f9c26a0093414d0d"} Jul 07 17:42:03.063633 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:03.063599 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfzd2" event={"ID":"786621af-8860-4858-a4a8-f9bc5ed7c9fe","Type":"ContainerStarted","Data":"47cd1318eb9b05d0d0cc85087fd790062de543d9d3fb55644691e84b057c3b24"} Jul 07 17:42:03.064146 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:03.063653 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfzd2" event={"ID":"786621af-8860-4858-a4a8-f9bc5ed7c9fe","Type":"ContainerStarted","Data":"226da58b81002e3fb5027f3191aa8fc11edba6d03ad32c0dd9c36d8318ead8fb"} Jul 07 17:42:03.085526 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:03.085464 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tfzd2" podStartSLOduration=2.8533860730000002 podStartE2EDuration="4.08543999s" totalImagesPullingTime="1.151770329s" totalInitContainerRuntime="80.283588ms" isStatefulPod=true podCreationTimestamp="2026-07-07 17:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:42:03.083756193 +0000 UTC m=+127.123083097" watchObservedRunningTime="2026-07-07 17:42:03.08543999 +0000 UTC m=+127.124766896" Jul 07 17:42:06.084248 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:06.084205 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" event={"ID":"79351d81-5f50-4f49-b563-89e8bdb23890","Type":"ContainerStarted","Data":"6a3fb11914fbaf6bd20376586c1b882fedd41b14fccb040c72b855568269a388"} Jul 07 17:42:06.084637 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:06.084395 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" Jul 07 17:42:06.086320 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:06.086294 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" Jul 07 17:42:06.102890 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:06.102829 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b48497b46-ctzxk" podStartSLOduration=1.8704619139999998 podStartE2EDuration="23.102812266s" totalImagesPullingTime="21.232350352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:41:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:42:06.102579725 +0000 UTC m=+130.141906630" watchObservedRunningTime="2026-07-07 17:42:06.102812266 +0000 UTC m=+130.142139169" Jul 07 17:42:06.413356 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:06.413259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs\") pod \"network-metrics-daemon-jdmfc\" (UID: \"9541dd58-70fc-4069-bdba-f4726c87e4d2\") " pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:42:06.416071 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:06.416006 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9541dd58-70fc-4069-bdba-f4726c87e4d2-metrics-certs\") pod \"network-metrics-daemon-jdmfc\" (UID: \"9541dd58-70fc-4069-bdba-f4726c87e4d2\") " pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:42:06.546022 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:06.545991 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2hwnn\"" Jul 07 17:42:06.554170 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:06.554133 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jdmfc" Jul 07 17:42:06.573200 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:06.573125 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-675994669b-7mkgg"] Jul 07 17:42:06.701451 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:06.701114 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jdmfc"] Jul 07 17:42:06.704031 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:42:06.703997 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9541dd58_70fc_4069_bdba_f4726c87e4d2.slice/crio-59c59046a1e621bdcbb856e0084a8ffefd6c10ad0ffa3bc17dee4e4d2ef179a4 WatchSource:0}: Error finding container 59c59046a1e621bdcbb856e0084a8ffefd6c10ad0ffa3bc17dee4e4d2ef179a4: Status 404 returned error can't find the container with id 59c59046a1e621bdcbb856e0084a8ffefd6c10ad0ffa3bc17dee4e4d2ef179a4 Jul 07 17:42:07.088756 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:07.088647 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jdmfc" event={"ID":"9541dd58-70fc-4069-bdba-f4726c87e4d2","Type":"ContainerStarted","Data":"59c59046a1e621bdcbb856e0084a8ffefd6c10ad0ffa3bc17dee4e4d2ef179a4"} Jul 07 17:42:07.510794 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:07.510765 2577 scope.go:117] "RemoveContainer" containerID="4b848b614abb4f156182bba3fe142656eff19c1a35bcb56bcb62537a73f328dd" Jul 07 17:42:08.094243 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.094137 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jdmfc" event={"ID":"9541dd58-70fc-4069-bdba-f4726c87e4d2","Type":"ContainerStarted","Data":"439bd3954eb8e355634ef34f0b312ebf85824c6b16d8ece015094181c4c8494f"} Jul 07 17:42:08.094243 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.094203 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jdmfc" event={"ID":"9541dd58-70fc-4069-bdba-f4726c87e4d2","Type":"ContainerStarted","Data":"1f3f14733562f8cad79b8153e8b855d72887194b5be8b25f1a709e46b7e0cbf0"} Jul 07 17:42:08.096144 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.096122 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 17:42:08.096292 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.096239 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" event={"ID":"b473a0e9-cbb7-404f-9f76-94e3bc7394ca","Type":"ContainerStarted","Data":"e487ccfdf4b42f2a433004c7564d55958a0d0d284e4a8fa755244f3aee44e6d2"} Jul 07 17:42:08.096665 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.096639 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:42:08.112545 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.112489 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jdmfc" podStartSLOduration=130.077560677 podStartE2EDuration="2m11.112468862s" totalImagesPullingTime="1.034908185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:42:08.111516884 +0000 UTC m=+132.150843786" watchObservedRunningTime="2026-07-07 17:42:08.112468862 +0000 UTC m=+132.151795764" Jul 07 17:42:08.135655 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.135593 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" podStartSLOduration=54.844480313 podStartE2EDuration="59.135570738s" totalImagesPullingTime="4.291090425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:42:08.1340243 +0000 UTC m=+132.173351204" watchObservedRunningTime="2026-07-07 17:42:08.135570738 +0000 UTC m=+132.174897642" Jul 07 17:42:08.637874 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.637844 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-5brrn" Jul 07 17:42:08.830455 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.830416 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-dhg9p"] Jul 07 17:42:08.833774 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.833750 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-dhg9p" Jul 07 17:42:08.836341 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.836316 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 17:42:08.836489 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.836412 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 17:42:08.836785 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.836766 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-cg4x2\"" Jul 07 17:42:08.844309 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.844282 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-dhg9p"] Jul 07 17:42:08.933551 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:08.933506 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx7wp\" (UniqueName: \"kubernetes.io/projected/ebf1176b-397e-426a-af38-040422cbb630-kube-api-access-wx7wp\") pod \"downloads-685bfb5f64-dhg9p\" (UID: \"ebf1176b-397e-426a-af38-040422cbb630\") " pod="openshift-console/downloads-685bfb5f64-dhg9p" Jul 07 17:42:09.034060 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:09.034018 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx7wp\" (UniqueName: \"kubernetes.io/projected/ebf1176b-397e-426a-af38-040422cbb630-kube-api-access-wx7wp\") pod \"downloads-685bfb5f64-dhg9p\" (UID: \"ebf1176b-397e-426a-af38-040422cbb630\") " pod="openshift-console/downloads-685bfb5f64-dhg9p" Jul 07 17:42:09.042903 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:09.042862 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx7wp\" (UniqueName: \"kubernetes.io/projected/ebf1176b-397e-426a-af38-040422cbb630-kube-api-access-wx7wp\") pod \"downloads-685bfb5f64-dhg9p\" (UID: \"ebf1176b-397e-426a-af38-040422cbb630\") " pod="openshift-console/downloads-685bfb5f64-dhg9p" Jul 07 17:42:09.144670 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:09.144625 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-dhg9p" Jul 07 17:42:09.288623 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:09.288596 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-dhg9p"] Jul 07 17:42:09.291551 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:42:09.291520 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebf1176b_397e_426a_af38_040422cbb630.slice/crio-5cb7e8884d1dc0ceaf0aac6597a314f0ae4d3f6f8dfcacfb534705fc50e445d3 WatchSource:0}: Error finding container 5cb7e8884d1dc0ceaf0aac6597a314f0ae4d3f6f8dfcacfb534705fc50e445d3: Status 404 returned error can't find the container with id 5cb7e8884d1dc0ceaf0aac6597a314f0ae4d3f6f8dfcacfb534705fc50e445d3 Jul 07 17:42:10.106536 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:10.106493 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-dhg9p" event={"ID":"ebf1176b-397e-426a-af38-040422cbb630","Type":"ContainerStarted","Data":"5cb7e8884d1dc0ceaf0aac6597a314f0ae4d3f6f8dfcacfb534705fc50e445d3"} Jul 07 17:42:18.370701 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.370658 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7974c855f9-l7mkz"] Jul 07 17:42:18.377833 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.377803 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.381366 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.381339 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 17:42:18.383437 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.383246 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 17:42:18.383437 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.383409 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 17:42:18.383437 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.383427 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 17:42:18.385146 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.385123 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 17:42:18.385358 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.385337 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-892kd\"" Jul 07 17:42:18.404777 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.404742 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7974c855f9-l7mkz"] Jul 07 17:42:18.520667 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.520634 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99rqr\" (UniqueName: \"kubernetes.io/projected/4c2a1d1b-1c77-47a1-a769-61999989482f-kube-api-access-99rqr\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.520887 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.520700 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-oauth-serving-cert\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.520887 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.520765 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-service-ca\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.520887 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.520834 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c2a1d1b-1c77-47a1-a769-61999989482f-console-serving-cert\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.520887 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.520873 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c2a1d1b-1c77-47a1-a769-61999989482f-console-oauth-config\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.521120 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.520893 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-console-config\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.622896 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.622789 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-service-ca\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.622896 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.622855 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c2a1d1b-1c77-47a1-a769-61999989482f-console-serving-cert\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.622896 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.622897 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c2a1d1b-1c77-47a1-a769-61999989482f-console-oauth-config\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.623250 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.622927 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-console-config\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.623250 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.622967 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99rqr\" (UniqueName: \"kubernetes.io/projected/4c2a1d1b-1c77-47a1-a769-61999989482f-kube-api-access-99rqr\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.623250 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.623024 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-oauth-serving-cert\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.624344 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.623938 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-oauth-serving-cert\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.628075 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.624481 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-service-ca\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.628075 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.625349 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-console-config\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.628075 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.626953 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c2a1d1b-1c77-47a1-a769-61999989482f-console-oauth-config\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.628075 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.627205 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c2a1d1b-1c77-47a1-a769-61999989482f-console-serving-cert\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.635609 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.635573 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99rqr\" (UniqueName: \"kubernetes.io/projected/4c2a1d1b-1c77-47a1-a769-61999989482f-kube-api-access-99rqr\") pod \"console-7974c855f9-l7mkz\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.690632 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.690589 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:18.879848 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:18.879754 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7974c855f9-l7mkz"] Jul 07 17:42:18.883783 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:42:18.883747 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c2a1d1b_1c77_47a1_a769_61999989482f.slice/crio-8c7b5577c03f77591067663be8536212709e8f7fd2768d41985ac664f457be80 WatchSource:0}: Error finding container 8c7b5577c03f77591067663be8536212709e8f7fd2768d41985ac664f457be80: Status 404 returned error can't find the container with id 8c7b5577c03f77591067663be8536212709e8f7fd2768d41985ac664f457be80 Jul 07 17:42:19.147406 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:19.147311 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7974c855f9-l7mkz" event={"ID":"4c2a1d1b-1c77-47a1-a769-61999989482f","Type":"ContainerStarted","Data":"8c7b5577c03f77591067663be8536212709e8f7fd2768d41985ac664f457be80"} Jul 07 17:42:20.153609 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:20.153513 2577 generic.go:356] "Generic (PLEG): container finished" podID="55437412-4864-499e-82c2-a854b002bb47" containerID="0a3a53149f6acd8fc2c9bd37391fe0ed0ca3ea96f0fbdecfda64208eb1ff5c4e" exitCode=0 Jul 07 17:42:20.153609 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:20.153598 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" event={"ID":"55437412-4864-499e-82c2-a854b002bb47","Type":"ContainerDied","Data":"0a3a53149f6acd8fc2c9bd37391fe0ed0ca3ea96f0fbdecfda64208eb1ff5c4e"} Jul 07 17:42:20.154188 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:20.154022 2577 scope.go:117] "RemoveContainer" containerID="0a3a53149f6acd8fc2c9bd37391fe0ed0ca3ea96f0fbdecfda64208eb1ff5c4e" Jul 07 17:42:21.162895 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:21.162708 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dwjq4" event={"ID":"55437412-4864-499e-82c2-a854b002bb47","Type":"ContainerStarted","Data":"f2c6d7d950242877eee00222edb96e224e52102bb1798d65b63c4a81406c0a8b"} Jul 07 17:42:22.541512 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.541473 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cdd9ff954-zx24w"] Jul 07 17:42:22.545172 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.545144 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.554383 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.554352 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 17:42:22.556148 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.556077 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cdd9ff954-zx24w"] Jul 07 17:42:22.665626 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.665584 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a80aa053-2439-4f3c-8708-4e50294c5c23-console-serving-cert\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.665822 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.665635 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-service-ca\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.665822 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.665675 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-trusted-ca-bundle\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.665822 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.665728 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a80aa053-2439-4f3c-8708-4e50294c5c23-console-oauth-config\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.665822 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.665753 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-console-config\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.665822 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.665818 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tvgh\" (UniqueName: \"kubernetes.io/projected/a80aa053-2439-4f3c-8708-4e50294c5c23-kube-api-access-7tvgh\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.666098 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.665873 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-oauth-serving-cert\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.767383 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.767328 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tvgh\" (UniqueName: \"kubernetes.io/projected/a80aa053-2439-4f3c-8708-4e50294c5c23-kube-api-access-7tvgh\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.767576 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.767402 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-oauth-serving-cert\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.767576 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.767432 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a80aa053-2439-4f3c-8708-4e50294c5c23-console-serving-cert\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.767576 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.767454 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-service-ca\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.767576 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.767485 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-trusted-ca-bundle\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.767576 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.767547 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a80aa053-2439-4f3c-8708-4e50294c5c23-console-oauth-config\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.767576 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.767572 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-console-config\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.768481 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.768410 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-service-ca\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.768620 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.768528 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-oauth-serving-cert\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.768620 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.768552 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-console-config\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.769470 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.769421 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-trusted-ca-bundle\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.770613 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.770586 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a80aa053-2439-4f3c-8708-4e50294c5c23-console-oauth-config\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.770770 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.770745 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a80aa053-2439-4f3c-8708-4e50294c5c23-console-serving-cert\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.777183 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.777148 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tvgh\" (UniqueName: \"kubernetes.io/projected/a80aa053-2439-4f3c-8708-4e50294c5c23-kube-api-access-7tvgh\") pod \"console-7cdd9ff954-zx24w\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:22.858793 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:22.858691 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:29.608493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:29.608452 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cdd9ff954-zx24w"] Jul 07 17:42:29.611679 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:42:29.611644 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda80aa053_2439_4f3c_8708_4e50294c5c23.slice/crio-32401e9bed1f1d5815bbe0cbbb1b5a1e352b3a5c7af1831ac4776000e92bbdec WatchSource:0}: Error finding container 32401e9bed1f1d5815bbe0cbbb1b5a1e352b3a5c7af1831ac4776000e92bbdec: Status 404 returned error can't find the container with id 32401e9bed1f1d5815bbe0cbbb1b5a1e352b3a5c7af1831ac4776000e92bbdec Jul 07 17:42:30.204896 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:30.204798 2577 generic.go:356] "Generic (PLEG): container finished" podID="055ae3eb-05b3-4509-9008-d62baf858fb3" containerID="fe532d08ec833c3243726ce279c11c50dcf236d639534f1a3077be7d708a7231" exitCode=0 Jul 07 17:42:30.205120 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:30.204882 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" event={"ID":"055ae3eb-05b3-4509-9008-d62baf858fb3","Type":"ContainerDied","Data":"fe532d08ec833c3243726ce279c11c50dcf236d639534f1a3077be7d708a7231"} Jul 07 17:42:30.205408 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:30.205381 2577 scope.go:117] "RemoveContainer" containerID="fe532d08ec833c3243726ce279c11c50dcf236d639534f1a3077be7d708a7231" Jul 07 17:42:30.207125 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:30.207090 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-dhg9p" event={"ID":"ebf1176b-397e-426a-af38-040422cbb630","Type":"ContainerStarted","Data":"2221fadf809d96757c45a6f1581f03cc8d4428170500b93cfe9f12113b247352"} Jul 07 17:42:30.207244 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:30.207164 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-dhg9p" Jul 07 17:42:30.210177 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:30.210143 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7974c855f9-l7mkz" event={"ID":"4c2a1d1b-1c77-47a1-a769-61999989482f","Type":"ContainerStarted","Data":"178379a698a1084102e374a28cd4baa68ac62ba2cade842c25673fd0f85f643e"} Jul 07 17:42:30.211984 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:30.211954 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdd9ff954-zx24w" event={"ID":"a80aa053-2439-4f3c-8708-4e50294c5c23","Type":"ContainerStarted","Data":"597ae2811ee23229b0277b259e5e49ad1f4acfffed2420966706b2ee42883979"} Jul 07 17:42:30.212139 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:30.212009 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdd9ff954-zx24w" event={"ID":"a80aa053-2439-4f3c-8708-4e50294c5c23","Type":"ContainerStarted","Data":"32401e9bed1f1d5815bbe0cbbb1b5a1e352b3a5c7af1831ac4776000e92bbdec"} Jul 07 17:42:30.215273 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:30.215247 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-dhg9p" Jul 07 17:42:30.239074 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:30.238967 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cdd9ff954-zx24w" podStartSLOduration=8.238944157 podStartE2EDuration="8.238944157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:42:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:42:30.238173782 +0000 UTC m=+154.277500686" watchObservedRunningTime="2026-07-07 17:42:30.238944157 +0000 UTC m=+154.278271063" Jul 07 17:42:30.296027 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:30.292404 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7974c855f9-l7mkz" podStartSLOduration=1.7201815580000002 podStartE2EDuration="12.292376652s" totalImagesPullingTime="10.572195094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:42:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:42:30.29130778 +0000 UTC m=+154.330634685" watchObservedRunningTime="2026-07-07 17:42:30.292376652 +0000 UTC m=+154.331703555" Jul 07 17:42:30.296027 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:30.292802 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-dhg9p" podStartSLOduration=2.077244663 podStartE2EDuration="22.292792147s" totalImagesPullingTime="20.215547484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:42:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:42:30.256709783 +0000 UTC m=+154.296036686" watchObservedRunningTime="2026-07-07 17:42:30.292792147 +0000 UTC m=+154.332119068" Jul 07 17:42:31.219672 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.219627 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2c8ld" event={"ID":"055ae3eb-05b3-4509-9008-d62baf858fb3","Type":"ContainerStarted","Data":"9a1fe354141aef8b3c921cca7dc785579eb45743f3c7f6c08930233a030fce12"} Jul 07 17:42:31.600605 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.600505 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-675994669b-7mkgg" podUID="89619451-c733-4743-90b3-ec2cffa338fc" containerName="registry" containerID="cri-o://34eb29a02eb6a21d4be299741265a1086fea89184d46f41ad360af352d0542fd" gracePeriod=30 Jul 07 17:42:31.930765 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.930728 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:42:31.958218 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.958178 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/89619451-c733-4743-90b3-ec2cffa338fc-trusted-ca\") pod \"89619451-c733-4743-90b3-ec2cffa338fc\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " Jul 07 17:42:31.958415 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.958245 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/89619451-c733-4743-90b3-ec2cffa338fc-registry-certificates\") pod \"89619451-c733-4743-90b3-ec2cffa338fc\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " Jul 07 17:42:31.958415 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.958295 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/89619451-c733-4743-90b3-ec2cffa338fc-ca-trust-extracted\") pod \"89619451-c733-4743-90b3-ec2cffa338fc\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " Jul 07 17:42:31.958415 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.958321 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls\") pod \"89619451-c733-4743-90b3-ec2cffa338fc\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " Jul 07 17:42:31.958415 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.958357 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-bound-sa-token\") pod \"89619451-c733-4743-90b3-ec2cffa338fc\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " Jul 07 17:42:31.958415 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.958389 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq5cc\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-kube-api-access-hq5cc\") pod \"89619451-c733-4743-90b3-ec2cffa338fc\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " Jul 07 17:42:31.958415 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.958411 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/89619451-c733-4743-90b3-ec2cffa338fc-installation-pull-secrets\") pod \"89619451-c733-4743-90b3-ec2cffa338fc\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " Jul 07 17:42:31.958695 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.958456 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/89619451-c733-4743-90b3-ec2cffa338fc-image-registry-private-configuration\") pod \"89619451-c733-4743-90b3-ec2cffa338fc\" (UID: \"89619451-c733-4743-90b3-ec2cffa338fc\") " Jul 07 17:42:31.959895 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.959811 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89619451-c733-4743-90b3-ec2cffa338fc-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "89619451-c733-4743-90b3-ec2cffa338fc" (UID: "89619451-c733-4743-90b3-ec2cffa338fc"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:42:31.961322 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.960904 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89619451-c733-4743-90b3-ec2cffa338fc-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "89619451-c733-4743-90b3-ec2cffa338fc" (UID: "89619451-c733-4743-90b3-ec2cffa338fc"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:42:31.962092 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.961851 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89619451-c733-4743-90b3-ec2cffa338fc-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "89619451-c733-4743-90b3-ec2cffa338fc" (UID: "89619451-c733-4743-90b3-ec2cffa338fc"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:42:31.962092 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.961896 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-kube-api-access-hq5cc" (OuterVolumeSpecName: "kube-api-access-hq5cc") pod "89619451-c733-4743-90b3-ec2cffa338fc" (UID: "89619451-c733-4743-90b3-ec2cffa338fc"). InnerVolumeSpecName "kube-api-access-hq5cc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:42:31.962092 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.961950 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "89619451-c733-4743-90b3-ec2cffa338fc" (UID: "89619451-c733-4743-90b3-ec2cffa338fc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:42:31.963654 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.963381 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "89619451-c733-4743-90b3-ec2cffa338fc" (UID: "89619451-c733-4743-90b3-ec2cffa338fc"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:42:31.964162 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.964126 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89619451-c733-4743-90b3-ec2cffa338fc-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "89619451-c733-4743-90b3-ec2cffa338fc" (UID: "89619451-c733-4743-90b3-ec2cffa338fc"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:42:31.972286 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:31.972235 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89619451-c733-4743-90b3-ec2cffa338fc-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "89619451-c733-4743-90b3-ec2cffa338fc" (UID: "89619451-c733-4743-90b3-ec2cffa338fc"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:42:32.059363 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.059324 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq5cc\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-kube-api-access-hq5cc\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:42:32.059363 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.059363 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/89619451-c733-4743-90b3-ec2cffa338fc-installation-pull-secrets\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:42:32.059599 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.059377 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/89619451-c733-4743-90b3-ec2cffa338fc-image-registry-private-configuration\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:42:32.059599 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.059391 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/89619451-c733-4743-90b3-ec2cffa338fc-trusted-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:42:32.059599 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.059403 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/89619451-c733-4743-90b3-ec2cffa338fc-registry-certificates\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:42:32.059599 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.059414 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/89619451-c733-4743-90b3-ec2cffa338fc-ca-trust-extracted\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:42:32.059599 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.059425 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-registry-tls\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:42:32.059599 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.059436 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89619451-c733-4743-90b3-ec2cffa338fc-bound-sa-token\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:42:32.225911 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.225870 2577 generic.go:356] "Generic (PLEG): container finished" podID="89619451-c733-4743-90b3-ec2cffa338fc" containerID="34eb29a02eb6a21d4be299741265a1086fea89184d46f41ad360af352d0542fd" exitCode=0 Jul 07 17:42:32.226460 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.225929 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-675994669b-7mkgg" Jul 07 17:42:32.226460 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.225961 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-675994669b-7mkgg" event={"ID":"89619451-c733-4743-90b3-ec2cffa338fc","Type":"ContainerDied","Data":"34eb29a02eb6a21d4be299741265a1086fea89184d46f41ad360af352d0542fd"} Jul 07 17:42:32.226460 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.226094 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-675994669b-7mkgg" event={"ID":"89619451-c733-4743-90b3-ec2cffa338fc","Type":"ContainerDied","Data":"dae6a14f5bd936b6125811bc092d495b9b1929fae7340e10f02b0713dd6d4038"} Jul 07 17:42:32.226460 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.226125 2577 scope.go:117] "RemoveContainer" containerID="34eb29a02eb6a21d4be299741265a1086fea89184d46f41ad360af352d0542fd" Jul 07 17:42:32.241822 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.241799 2577 scope.go:117] "RemoveContainer" containerID="34eb29a02eb6a21d4be299741265a1086fea89184d46f41ad360af352d0542fd" Jul 07 17:42:32.242289 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:42:32.242256 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34eb29a02eb6a21d4be299741265a1086fea89184d46f41ad360af352d0542fd\": container with ID starting with 34eb29a02eb6a21d4be299741265a1086fea89184d46f41ad360af352d0542fd not found: ID does not exist" containerID="34eb29a02eb6a21d4be299741265a1086fea89184d46f41ad360af352d0542fd" Jul 07 17:42:32.242386 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.242302 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34eb29a02eb6a21d4be299741265a1086fea89184d46f41ad360af352d0542fd"} err="failed to get container status \"34eb29a02eb6a21d4be299741265a1086fea89184d46f41ad360af352d0542fd\": rpc error: code = NotFound desc = could not find container \"34eb29a02eb6a21d4be299741265a1086fea89184d46f41ad360af352d0542fd\": container with ID starting with 34eb29a02eb6a21d4be299741265a1086fea89184d46f41ad360af352d0542fd not found: ID does not exist" Jul 07 17:42:32.249465 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.249421 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-675994669b-7mkgg"] Jul 07 17:42:32.253860 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.253811 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-675994669b-7mkgg"] Jul 07 17:42:32.254098 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.254074 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-jzjvx_2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4/cluster-monitoring-operator/0.log" Jul 07 17:42:32.516930 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.516852 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89619451-c733-4743-90b3-ec2cffa338fc" path="/var/lib/kubelet/pods/89619451-c733-4743-90b3-ec2cffa338fc/volumes" Jul 07 17:42:32.859309 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.859209 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:32.859309 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.859293 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:32.865250 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:32.865216 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:33.238103 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:33.238066 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:42:33.281441 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:33.281407 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7974c855f9-l7mkz"] Jul 07 17:42:34.052121 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:34.052088 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfzd2_786621af-8860-4858-a4a8-f9bc5ed7c9fe/init-textfile/0.log" Jul 07 17:42:34.262196 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:34.262162 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfzd2_786621af-8860-4858-a4a8-f9bc5ed7c9fe/node-exporter/0.log" Jul 07 17:42:34.450769 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:34.450718 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfzd2_786621af-8860-4858-a4a8-f9bc5ed7c9fe/kube-rbac-proxy/0.log" Jul 07 17:42:38.691010 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:38.690961 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:39.250452 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:39.250424 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 17:42:39.453360 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:39.453329 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/3.log" Jul 07 17:42:39.651780 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:39.651691 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7974c855f9-l7mkz_4c2a1d1b-1c77-47a1-a769-61999989482f/console/0.log" Jul 07 17:42:39.851552 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:39.851522 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7cdd9ff954-zx24w_a80aa053-2439-4f3c-8708-4e50294c5c23/console/0.log" Jul 07 17:42:40.053226 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:40.053195 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-dhg9p_ebf1176b-397e-426a-af38-040422cbb630/download-server/0.log" Jul 07 17:42:40.264927 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:40.264844 2577 generic.go:356] "Generic (PLEG): container finished" podID="d707cdcf-3720-438d-a034-cf5dd54f2a47" containerID="e806035eeb9752c5030e1bed4de9819a05930af57202993328969f3cd6eb9ab8" exitCode=0 Jul 07 17:42:40.264927 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:40.264915 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" event={"ID":"d707cdcf-3720-438d-a034-cf5dd54f2a47","Type":"ContainerDied","Data":"e806035eeb9752c5030e1bed4de9819a05930af57202993328969f3cd6eb9ab8"} Jul 07 17:42:40.265284 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:40.265269 2577 scope.go:117] "RemoveContainer" containerID="e806035eeb9752c5030e1bed4de9819a05930af57202993328969f3cd6eb9ab8" Jul 07 17:42:41.270947 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:41.270910 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-w8smz" event={"ID":"d707cdcf-3720-438d-a034-cf5dd54f2a47","Type":"ContainerStarted","Data":"1f89ddbf37bdd015a2ef4bffdbe02fb10d14c2ff496ac8cccaf6fffd1104069d"} Jul 07 17:42:58.309276 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.309225 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7974c855f9-l7mkz" podUID="4c2a1d1b-1c77-47a1-a769-61999989482f" containerName="console" containerID="cri-o://178379a698a1084102e374a28cd4baa68ac62ba2cade842c25673fd0f85f643e" gracePeriod=15 Jul 07 17:42:58.601241 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.601204 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:58.699183 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.699149 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-oauth-serving-cert\") pod \"4c2a1d1b-1c77-47a1-a769-61999989482f\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " Jul 07 17:42:58.699183 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.699201 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c2a1d1b-1c77-47a1-a769-61999989482f-console-serving-cert\") pod \"4c2a1d1b-1c77-47a1-a769-61999989482f\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " Jul 07 17:42:58.699428 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.699254 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-console-config\") pod \"4c2a1d1b-1c77-47a1-a769-61999989482f\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " Jul 07 17:42:58.699428 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.699270 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c2a1d1b-1c77-47a1-a769-61999989482f-console-oauth-config\") pod \"4c2a1d1b-1c77-47a1-a769-61999989482f\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " Jul 07 17:42:58.699428 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.699300 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99rqr\" (UniqueName: \"kubernetes.io/projected/4c2a1d1b-1c77-47a1-a769-61999989482f-kube-api-access-99rqr\") pod \"4c2a1d1b-1c77-47a1-a769-61999989482f\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " Jul 07 17:42:58.699428 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.699352 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-service-ca\") pod \"4c2a1d1b-1c77-47a1-a769-61999989482f\" (UID: \"4c2a1d1b-1c77-47a1-a769-61999989482f\") " Jul 07 17:42:58.699775 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.699741 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4c2a1d1b-1c77-47a1-a769-61999989482f" (UID: "4c2a1d1b-1c77-47a1-a769-61999989482f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:42:58.699775 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.699768 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-console-config" (OuterVolumeSpecName: "console-config") pod "4c2a1d1b-1c77-47a1-a769-61999989482f" (UID: "4c2a1d1b-1c77-47a1-a769-61999989482f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:42:58.699910 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.699821 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-service-ca" (OuterVolumeSpecName: "service-ca") pod "4c2a1d1b-1c77-47a1-a769-61999989482f" (UID: "4c2a1d1b-1c77-47a1-a769-61999989482f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:42:58.701735 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.701699 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c2a1d1b-1c77-47a1-a769-61999989482f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4c2a1d1b-1c77-47a1-a769-61999989482f" (UID: "4c2a1d1b-1c77-47a1-a769-61999989482f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:42:58.701873 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.701749 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c2a1d1b-1c77-47a1-a769-61999989482f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4c2a1d1b-1c77-47a1-a769-61999989482f" (UID: "4c2a1d1b-1c77-47a1-a769-61999989482f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:42:58.701873 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.701800 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c2a1d1b-1c77-47a1-a769-61999989482f-kube-api-access-99rqr" (OuterVolumeSpecName: "kube-api-access-99rqr") pod "4c2a1d1b-1c77-47a1-a769-61999989482f" (UID: "4c2a1d1b-1c77-47a1-a769-61999989482f"). InnerVolumeSpecName "kube-api-access-99rqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:42:58.800852 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.800815 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-console-config\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:42:58.800852 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.800850 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c2a1d1b-1c77-47a1-a769-61999989482f-console-oauth-config\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:42:58.800852 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.800861 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99rqr\" (UniqueName: \"kubernetes.io/projected/4c2a1d1b-1c77-47a1-a769-61999989482f-kube-api-access-99rqr\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:42:58.801144 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.800870 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:42:58.801144 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.800879 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c2a1d1b-1c77-47a1-a769-61999989482f-oauth-serving-cert\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:42:58.801144 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:58.800889 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c2a1d1b-1c77-47a1-a769-61999989482f-console-serving-cert\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:42:59.345755 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:59.345721 2577 generic.go:356] "Generic (PLEG): container finished" podID="4c2a1d1b-1c77-47a1-a769-61999989482f" containerID="178379a698a1084102e374a28cd4baa68ac62ba2cade842c25673fd0f85f643e" exitCode=0 Jul 07 17:42:59.346251 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:59.345766 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7974c855f9-l7mkz" event={"ID":"4c2a1d1b-1c77-47a1-a769-61999989482f","Type":"ContainerDied","Data":"178379a698a1084102e374a28cd4baa68ac62ba2cade842c25673fd0f85f643e"} Jul 07 17:42:59.346251 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:59.345770 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7974c855f9-l7mkz" Jul 07 17:42:59.346251 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:59.345802 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7974c855f9-l7mkz" event={"ID":"4c2a1d1b-1c77-47a1-a769-61999989482f","Type":"ContainerDied","Data":"8c7b5577c03f77591067663be8536212709e8f7fd2768d41985ac664f457be80"} Jul 07 17:42:59.346251 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:59.345821 2577 scope.go:117] "RemoveContainer" containerID="178379a698a1084102e374a28cd4baa68ac62ba2cade842c25673fd0f85f643e" Jul 07 17:42:59.357800 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:59.357771 2577 scope.go:117] "RemoveContainer" containerID="178379a698a1084102e374a28cd4baa68ac62ba2cade842c25673fd0f85f643e" Jul 07 17:42:59.358196 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:42:59.358175 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"178379a698a1084102e374a28cd4baa68ac62ba2cade842c25673fd0f85f643e\": container with ID starting with 178379a698a1084102e374a28cd4baa68ac62ba2cade842c25673fd0f85f643e not found: ID does not exist" containerID="178379a698a1084102e374a28cd4baa68ac62ba2cade842c25673fd0f85f643e" Jul 07 17:42:59.358258 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:59.358206 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"178379a698a1084102e374a28cd4baa68ac62ba2cade842c25673fd0f85f643e"} err="failed to get container status \"178379a698a1084102e374a28cd4baa68ac62ba2cade842c25673fd0f85f643e\": rpc error: code = NotFound desc = could not find container \"178379a698a1084102e374a28cd4baa68ac62ba2cade842c25673fd0f85f643e\": container with ID starting with 178379a698a1084102e374a28cd4baa68ac62ba2cade842c25673fd0f85f643e not found: ID does not exist" Jul 07 17:42:59.370023 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:59.369984 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7974c855f9-l7mkz"] Jul 07 17:42:59.373760 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:42:59.373729 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7974c855f9-l7mkz"] Jul 07 17:43:00.515661 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:00.515626 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c2a1d1b-1c77-47a1-a769-61999989482f" path="/var/lib/kubelet/pods/4c2a1d1b-1c77-47a1-a769-61999989482f/volumes" Jul 07 17:43:28.789383 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.789338 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56b9c4d87b-lxq89"] Jul 07 17:43:28.789900 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.789661 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c2a1d1b-1c77-47a1-a769-61999989482f" containerName="console" Jul 07 17:43:28.789900 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.789676 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c2a1d1b-1c77-47a1-a769-61999989482f" containerName="console" Jul 07 17:43:28.789900 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.789698 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89619451-c733-4743-90b3-ec2cffa338fc" containerName="registry" Jul 07 17:43:28.789900 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.789703 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="89619451-c733-4743-90b3-ec2cffa338fc" containerName="registry" Jul 07 17:43:28.789900 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.789770 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="89619451-c733-4743-90b3-ec2cffa338fc" containerName="registry" Jul 07 17:43:28.789900 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.789781 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c2a1d1b-1c77-47a1-a769-61999989482f" containerName="console" Jul 07 17:43:28.791803 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.791778 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.805172 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.805141 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56b9c4d87b-lxq89"] Jul 07 17:43:28.825142 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.825108 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17c91427-59b2-4c7f-ae57-824dbfa04949-console-oauth-config\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.825320 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.825152 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17c91427-59b2-4c7f-ae57-824dbfa04949-console-serving-cert\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.825320 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.825178 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-oauth-serving-cert\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.825320 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.825252 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkbxj\" (UniqueName: \"kubernetes.io/projected/17c91427-59b2-4c7f-ae57-824dbfa04949-kube-api-access-fkbxj\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.825320 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.825293 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-service-ca\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.825454 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.825327 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-trusted-ca-bundle\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.825454 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.825346 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-console-config\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.926516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.926469 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17c91427-59b2-4c7f-ae57-824dbfa04949-console-oauth-config\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.926516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.926521 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17c91427-59b2-4c7f-ae57-824dbfa04949-console-serving-cert\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.926780 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.926548 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-oauth-serving-cert\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.926780 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.926583 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkbxj\" (UniqueName: \"kubernetes.io/projected/17c91427-59b2-4c7f-ae57-824dbfa04949-kube-api-access-fkbxj\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.926780 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.926627 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-service-ca\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.926780 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.926663 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-trusted-ca-bundle\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.926780 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.926689 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-console-config\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.927424 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.927392 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-service-ca\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.927647 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.927616 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-trusted-ca-bundle\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.927720 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.927655 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-console-config\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.927720 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.927695 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-oauth-serving-cert\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.929378 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.929350 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17c91427-59b2-4c7f-ae57-824dbfa04949-console-serving-cert\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.929475 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.929379 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17c91427-59b2-4c7f-ae57-824dbfa04949-console-oauth-config\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:28.935649 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:28.935620 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkbxj\" (UniqueName: \"kubernetes.io/projected/17c91427-59b2-4c7f-ae57-824dbfa04949-kube-api-access-fkbxj\") pod \"console-56b9c4d87b-lxq89\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:29.105769 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:29.105672 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:29.252348 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:29.252324 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56b9c4d87b-lxq89"] Jul 07 17:43:29.255202 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:43:29.255171 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17c91427_59b2_4c7f_ae57_824dbfa04949.slice/crio-03d8b8bee021309f6f37b9c42444593cf16c0cb669b67993f488f42cedac030a WatchSource:0}: Error finding container 03d8b8bee021309f6f37b9c42444593cf16c0cb669b67993f488f42cedac030a: Status 404 returned error can't find the container with id 03d8b8bee021309f6f37b9c42444593cf16c0cb669b67993f488f42cedac030a Jul 07 17:43:29.463077 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:29.463007 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b9c4d87b-lxq89" event={"ID":"17c91427-59b2-4c7f-ae57-824dbfa04949","Type":"ContainerStarted","Data":"b0ba1f17575ad90af8a29555418ffd747af4057b1ab77841eecef253e13b0fc9"} Jul 07 17:43:29.463275 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:29.463096 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b9c4d87b-lxq89" event={"ID":"17c91427-59b2-4c7f-ae57-824dbfa04949","Type":"ContainerStarted","Data":"03d8b8bee021309f6f37b9c42444593cf16c0cb669b67993f488f42cedac030a"} Jul 07 17:43:29.482796 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:29.482734 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56b9c4d87b-lxq89" podStartSLOduration=1.482718458 podStartE2EDuration="1.482718458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:43:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:43:29.480752112 +0000 UTC m=+213.520079016" watchObservedRunningTime="2026-07-07 17:43:29.482718458 +0000 UTC m=+213.522045362" Jul 07 17:43:39.106287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:39.106240 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:39.106735 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:39.106319 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:39.111201 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:39.111178 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:39.505942 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:39.505911 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:43:39.553520 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:43:39.553479 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cdd9ff954-zx24w"] Jul 07 17:44:04.582619 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:04.582549 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7cdd9ff954-zx24w" podUID="a80aa053-2439-4f3c-8708-4e50294c5c23" containerName="console" containerID="cri-o://597ae2811ee23229b0277b259e5e49ad1f4acfffed2420966706b2ee42883979" gracePeriod=15 Jul 07 17:44:04.848657 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:04.848621 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:44:05.027946 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.027908 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tvgh\" (UniqueName: \"kubernetes.io/projected/a80aa053-2439-4f3c-8708-4e50294c5c23-kube-api-access-7tvgh\") pod \"a80aa053-2439-4f3c-8708-4e50294c5c23\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " Jul 07 17:44:05.028180 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.027970 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a80aa053-2439-4f3c-8708-4e50294c5c23-console-serving-cert\") pod \"a80aa053-2439-4f3c-8708-4e50294c5c23\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " Jul 07 17:44:05.028180 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.028003 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-console-config\") pod \"a80aa053-2439-4f3c-8708-4e50294c5c23\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " Jul 07 17:44:05.028180 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.028019 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-oauth-serving-cert\") pod \"a80aa053-2439-4f3c-8708-4e50294c5c23\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " Jul 07 17:44:05.028180 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.028069 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-trusted-ca-bundle\") pod \"a80aa053-2439-4f3c-8708-4e50294c5c23\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " Jul 07 17:44:05.028180 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.028113 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-service-ca\") pod \"a80aa053-2439-4f3c-8708-4e50294c5c23\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " Jul 07 17:44:05.028180 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.028137 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a80aa053-2439-4f3c-8708-4e50294c5c23-console-oauth-config\") pod \"a80aa053-2439-4f3c-8708-4e50294c5c23\" (UID: \"a80aa053-2439-4f3c-8708-4e50294c5c23\") " Jul 07 17:44:05.028596 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.028565 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-console-config" (OuterVolumeSpecName: "console-config") pod "a80aa053-2439-4f3c-8708-4e50294c5c23" (UID: "a80aa053-2439-4f3c-8708-4e50294c5c23"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:44:05.028596 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.028575 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-service-ca" (OuterVolumeSpecName: "service-ca") pod "a80aa053-2439-4f3c-8708-4e50294c5c23" (UID: "a80aa053-2439-4f3c-8708-4e50294c5c23"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:44:05.028713 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.028587 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a80aa053-2439-4f3c-8708-4e50294c5c23" (UID: "a80aa053-2439-4f3c-8708-4e50294c5c23"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:44:05.028713 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.028614 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a80aa053-2439-4f3c-8708-4e50294c5c23" (UID: "a80aa053-2439-4f3c-8708-4e50294c5c23"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:44:05.030523 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.030497 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a80aa053-2439-4f3c-8708-4e50294c5c23-kube-api-access-7tvgh" (OuterVolumeSpecName: "kube-api-access-7tvgh") pod "a80aa053-2439-4f3c-8708-4e50294c5c23" (UID: "a80aa053-2439-4f3c-8708-4e50294c5c23"). InnerVolumeSpecName "kube-api-access-7tvgh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:44:05.030639 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.030499 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a80aa053-2439-4f3c-8708-4e50294c5c23-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a80aa053-2439-4f3c-8708-4e50294c5c23" (UID: "a80aa053-2439-4f3c-8708-4e50294c5c23"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:44:05.030639 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.030542 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a80aa053-2439-4f3c-8708-4e50294c5c23-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a80aa053-2439-4f3c-8708-4e50294c5c23" (UID: "a80aa053-2439-4f3c-8708-4e50294c5c23"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:44:05.129267 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.129178 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:44:05.129267 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.129209 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a80aa053-2439-4f3c-8708-4e50294c5c23-console-oauth-config\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:44:05.129267 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.129219 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tvgh\" (UniqueName: \"kubernetes.io/projected/a80aa053-2439-4f3c-8708-4e50294c5c23-kube-api-access-7tvgh\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:44:05.129267 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.129228 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a80aa053-2439-4f3c-8708-4e50294c5c23-console-serving-cert\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:44:05.129267 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.129237 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-console-config\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:44:05.129267 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.129246 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-oauth-serving-cert\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:44:05.129267 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.129253 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a80aa053-2439-4f3c-8708-4e50294c5c23-trusted-ca-bundle\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:44:05.605151 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.605114 2577 generic.go:356] "Generic (PLEG): container finished" podID="a80aa053-2439-4f3c-8708-4e50294c5c23" containerID="597ae2811ee23229b0277b259e5e49ad1f4acfffed2420966706b2ee42883979" exitCode=0 Jul 07 17:44:05.605577 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.605167 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdd9ff954-zx24w" Jul 07 17:44:05.605577 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.605184 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdd9ff954-zx24w" event={"ID":"a80aa053-2439-4f3c-8708-4e50294c5c23","Type":"ContainerDied","Data":"597ae2811ee23229b0277b259e5e49ad1f4acfffed2420966706b2ee42883979"} Jul 07 17:44:05.605577 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.605227 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdd9ff954-zx24w" event={"ID":"a80aa053-2439-4f3c-8708-4e50294c5c23","Type":"ContainerDied","Data":"32401e9bed1f1d5815bbe0cbbb1b5a1e352b3a5c7af1831ac4776000e92bbdec"} Jul 07 17:44:05.605577 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.605249 2577 scope.go:117] "RemoveContainer" containerID="597ae2811ee23229b0277b259e5e49ad1f4acfffed2420966706b2ee42883979" Jul 07 17:44:05.614298 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.614277 2577 scope.go:117] "RemoveContainer" containerID="597ae2811ee23229b0277b259e5e49ad1f4acfffed2420966706b2ee42883979" Jul 07 17:44:05.614621 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:44:05.614603 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"597ae2811ee23229b0277b259e5e49ad1f4acfffed2420966706b2ee42883979\": container with ID starting with 597ae2811ee23229b0277b259e5e49ad1f4acfffed2420966706b2ee42883979 not found: ID does not exist" containerID="597ae2811ee23229b0277b259e5e49ad1f4acfffed2420966706b2ee42883979" Jul 07 17:44:05.614668 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.614632 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"597ae2811ee23229b0277b259e5e49ad1f4acfffed2420966706b2ee42883979"} err="failed to get container status \"597ae2811ee23229b0277b259e5e49ad1f4acfffed2420966706b2ee42883979\": rpc error: code = NotFound desc = could not find container \"597ae2811ee23229b0277b259e5e49ad1f4acfffed2420966706b2ee42883979\": container with ID starting with 597ae2811ee23229b0277b259e5e49ad1f4acfffed2420966706b2ee42883979 not found: ID does not exist" Jul 07 17:44:05.629112 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.629072 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cdd9ff954-zx24w"] Jul 07 17:44:05.635645 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:05.635610 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cdd9ff954-zx24w"] Jul 07 17:44:06.515862 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:06.515825 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a80aa053-2439-4f3c-8708-4e50294c5c23" path="/var/lib/kubelet/pods/a80aa053-2439-4f3c-8708-4e50294c5c23/volumes" Jul 07 17:44:35.331312 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.331224 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9df66c4db-8zcd2"] Jul 07 17:44:35.331978 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.331739 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a80aa053-2439-4f3c-8708-4e50294c5c23" containerName="console" Jul 07 17:44:35.331978 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.331760 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a80aa053-2439-4f3c-8708-4e50294c5c23" containerName="console" Jul 07 17:44:35.331978 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.331843 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a80aa053-2439-4f3c-8708-4e50294c5c23" containerName="console" Jul 07 17:44:35.335321 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.335288 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.348713 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.348679 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9df66c4db-8zcd2"] Jul 07 17:44:35.349348 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.349322 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-console-config\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.349491 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.349355 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-trusted-ca-bundle\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.349491 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.349380 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d15f2074-44cb-416f-b396-36faa59058d4-console-oauth-config\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.349563 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.349484 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-oauth-serving-cert\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.349563 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.349525 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d15f2074-44cb-416f-b396-36faa59058d4-console-serving-cert\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.349623 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.349565 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k8dd\" (UniqueName: \"kubernetes.io/projected/d15f2074-44cb-416f-b396-36faa59058d4-kube-api-access-9k8dd\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.349658 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.349619 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-service-ca\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.450194 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.450151 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-console-config\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.450194 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.450206 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-trusted-ca-bundle\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.450484 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.450331 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d15f2074-44cb-416f-b396-36faa59058d4-console-oauth-config\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.450484 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.450397 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-oauth-serving-cert\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.450484 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.450427 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d15f2074-44cb-416f-b396-36faa59058d4-console-serving-cert\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.450484 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.450443 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k8dd\" (UniqueName: \"kubernetes.io/projected/d15f2074-44cb-416f-b396-36faa59058d4-kube-api-access-9k8dd\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.450484 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.450465 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-service-ca\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.451107 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.451077 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-console-config\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.451238 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.451191 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-service-ca\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.451238 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.451226 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-trusted-ca-bundle\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.451424 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.451400 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-oauth-serving-cert\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.453302 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.453266 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d15f2074-44cb-416f-b396-36faa59058d4-console-oauth-config\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.453423 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.453266 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d15f2074-44cb-416f-b396-36faa59058d4-console-serving-cert\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.459646 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.459613 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k8dd\" (UniqueName: \"kubernetes.io/projected/d15f2074-44cb-416f-b396-36faa59058d4-kube-api-access-9k8dd\") pod \"console-9df66c4db-8zcd2\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.645499 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.645404 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:35.795661 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:35.795632 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9df66c4db-8zcd2"] Jul 07 17:44:35.798340 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:44:35.798311 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd15f2074_44cb_416f_b396_36faa59058d4.slice/crio-11f65f0280e950649954228413fc4be565518e962252f38e661d3dfea17271be WatchSource:0}: Error finding container 11f65f0280e950649954228413fc4be565518e962252f38e661d3dfea17271be: Status 404 returned error can't find the container with id 11f65f0280e950649954228413fc4be565518e962252f38e661d3dfea17271be Jul 07 17:44:36.729493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:36.729458 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9df66c4db-8zcd2" event={"ID":"d15f2074-44cb-416f-b396-36faa59058d4","Type":"ContainerStarted","Data":"df21aac42e8381c7f7f30341b219e0b9476bb6c7de00c7f865f4e6edb73aeee6"} Jul 07 17:44:36.729912 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:36.729513 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9df66c4db-8zcd2" event={"ID":"d15f2074-44cb-416f-b396-36faa59058d4","Type":"ContainerStarted","Data":"11f65f0280e950649954228413fc4be565518e962252f38e661d3dfea17271be"} Jul 07 17:44:36.746582 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:36.746523 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9df66c4db-8zcd2" podStartSLOduration=1.746505027 podStartE2EDuration="1.746505027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:44:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:44:36.746382614 +0000 UTC m=+280.785709542" watchObservedRunningTime="2026-07-07 17:44:36.746505027 +0000 UTC m=+280.785831930" Jul 07 17:44:45.646024 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:45.645986 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:45.646549 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:45.646092 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:45.651133 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:45.651102 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:45.768600 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:45.768564 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:44:45.827172 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:45.827131 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56b9c4d87b-lxq89"] Jul 07 17:44:56.408556 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:56.408495 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 17:44:56.418795 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:56.418761 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 17:44:56.423565 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:44:56.423537 2577 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 17:45:10.850568 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:10.850513 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-56b9c4d87b-lxq89" podUID="17c91427-59b2-4c7f-ae57-824dbfa04949" containerName="console" containerID="cri-o://b0ba1f17575ad90af8a29555418ffd747af4057b1ab77841eecef253e13b0fc9" gracePeriod=15 Jul 07 17:45:11.099211 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.099172 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:45:11.249776 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.249737 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-console-config\") pod \"17c91427-59b2-4c7f-ae57-824dbfa04949\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " Jul 07 17:45:11.249968 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.249806 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17c91427-59b2-4c7f-ae57-824dbfa04949-console-oauth-config\") pod \"17c91427-59b2-4c7f-ae57-824dbfa04949\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " Jul 07 17:45:11.249968 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.249823 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-oauth-serving-cert\") pod \"17c91427-59b2-4c7f-ae57-824dbfa04949\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " Jul 07 17:45:11.249968 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.249844 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-service-ca\") pod \"17c91427-59b2-4c7f-ae57-824dbfa04949\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " Jul 07 17:45:11.250142 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.249962 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-trusted-ca-bundle\") pod \"17c91427-59b2-4c7f-ae57-824dbfa04949\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " Jul 07 17:45:11.250142 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.250027 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17c91427-59b2-4c7f-ae57-824dbfa04949-console-serving-cert\") pod \"17c91427-59b2-4c7f-ae57-824dbfa04949\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " Jul 07 17:45:11.250142 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.250090 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkbxj\" (UniqueName: \"kubernetes.io/projected/17c91427-59b2-4c7f-ae57-824dbfa04949-kube-api-access-fkbxj\") pod \"17c91427-59b2-4c7f-ae57-824dbfa04949\" (UID: \"17c91427-59b2-4c7f-ae57-824dbfa04949\") " Jul 07 17:45:11.250367 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.250332 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-console-config" (OuterVolumeSpecName: "console-config") pod "17c91427-59b2-4c7f-ae57-824dbfa04949" (UID: "17c91427-59b2-4c7f-ae57-824dbfa04949"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:45:11.250492 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.250350 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-service-ca" (OuterVolumeSpecName: "service-ca") pod "17c91427-59b2-4c7f-ae57-824dbfa04949" (UID: "17c91427-59b2-4c7f-ae57-824dbfa04949"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:45:11.250492 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.250370 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "17c91427-59b2-4c7f-ae57-824dbfa04949" (UID: "17c91427-59b2-4c7f-ae57-824dbfa04949"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:45:11.250492 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.250469 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "17c91427-59b2-4c7f-ae57-824dbfa04949" (UID: "17c91427-59b2-4c7f-ae57-824dbfa04949"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:45:11.252437 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.252397 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17c91427-59b2-4c7f-ae57-824dbfa04949-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "17c91427-59b2-4c7f-ae57-824dbfa04949" (UID: "17c91427-59b2-4c7f-ae57-824dbfa04949"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:45:11.252551 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.252405 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17c91427-59b2-4c7f-ae57-824dbfa04949-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "17c91427-59b2-4c7f-ae57-824dbfa04949" (UID: "17c91427-59b2-4c7f-ae57-824dbfa04949"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:45:11.252551 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.252501 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17c91427-59b2-4c7f-ae57-824dbfa04949-kube-api-access-fkbxj" (OuterVolumeSpecName: "kube-api-access-fkbxj") pod "17c91427-59b2-4c7f-ae57-824dbfa04949" (UID: "17c91427-59b2-4c7f-ae57-824dbfa04949"). InnerVolumeSpecName "kube-api-access-fkbxj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:45:11.351167 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.351110 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-console-config\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:45:11.351167 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.351155 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17c91427-59b2-4c7f-ae57-824dbfa04949-console-oauth-config\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:45:11.351167 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.351167 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-oauth-serving-cert\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:45:11.351167 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.351176 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:45:11.351167 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.351184 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17c91427-59b2-4c7f-ae57-824dbfa04949-trusted-ca-bundle\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:45:11.351475 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.351194 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17c91427-59b2-4c7f-ae57-824dbfa04949-console-serving-cert\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:45:11.351475 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.351202 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkbxj\" (UniqueName: \"kubernetes.io/projected/17c91427-59b2-4c7f-ae57-824dbfa04949-kube-api-access-fkbxj\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:45:11.869641 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.869601 2577 generic.go:356] "Generic (PLEG): container finished" podID="17c91427-59b2-4c7f-ae57-824dbfa04949" containerID="b0ba1f17575ad90af8a29555418ffd747af4057b1ab77841eecef253e13b0fc9" exitCode=0 Jul 07 17:45:11.870106 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.869659 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b9c4d87b-lxq89" Jul 07 17:45:11.870106 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.869667 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b9c4d87b-lxq89" event={"ID":"17c91427-59b2-4c7f-ae57-824dbfa04949","Type":"ContainerDied","Data":"b0ba1f17575ad90af8a29555418ffd747af4057b1ab77841eecef253e13b0fc9"} Jul 07 17:45:11.870106 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.869706 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b9c4d87b-lxq89" event={"ID":"17c91427-59b2-4c7f-ae57-824dbfa04949","Type":"ContainerDied","Data":"03d8b8bee021309f6f37b9c42444593cf16c0cb669b67993f488f42cedac030a"} Jul 07 17:45:11.870106 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.869727 2577 scope.go:117] "RemoveContainer" containerID="b0ba1f17575ad90af8a29555418ffd747af4057b1ab77841eecef253e13b0fc9" Jul 07 17:45:11.878849 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.878826 2577 scope.go:117] "RemoveContainer" containerID="b0ba1f17575ad90af8a29555418ffd747af4057b1ab77841eecef253e13b0fc9" Jul 07 17:45:11.879213 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:45:11.879193 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0ba1f17575ad90af8a29555418ffd747af4057b1ab77841eecef253e13b0fc9\": container with ID starting with b0ba1f17575ad90af8a29555418ffd747af4057b1ab77841eecef253e13b0fc9 not found: ID does not exist" containerID="b0ba1f17575ad90af8a29555418ffd747af4057b1ab77841eecef253e13b0fc9" Jul 07 17:45:11.879296 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.879221 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0ba1f17575ad90af8a29555418ffd747af4057b1ab77841eecef253e13b0fc9"} err="failed to get container status \"b0ba1f17575ad90af8a29555418ffd747af4057b1ab77841eecef253e13b0fc9\": rpc error: code = NotFound desc = could not find container \"b0ba1f17575ad90af8a29555418ffd747af4057b1ab77841eecef253e13b0fc9\": container with ID starting with b0ba1f17575ad90af8a29555418ffd747af4057b1ab77841eecef253e13b0fc9 not found: ID does not exist" Jul 07 17:45:11.890712 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.890673 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56b9c4d87b-lxq89"] Jul 07 17:45:11.893700 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:11.893666 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56b9c4d87b-lxq89"] Jul 07 17:45:12.514990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:12.514950 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17c91427-59b2-4c7f-ae57-824dbfa04949" path="/var/lib/kubelet/pods/17c91427-59b2-4c7f-ae57-824dbfa04949/volumes" Jul 07 17:45:37.035315 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.035280 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc"] Jul 07 17:45:37.035713 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.035571 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17c91427-59b2-4c7f-ae57-824dbfa04949" containerName="console" Jul 07 17:45:37.035713 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.035581 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="17c91427-59b2-4c7f-ae57-824dbfa04949" containerName="console" Jul 07 17:45:37.035713 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.035630 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="17c91427-59b2-4c7f-ae57-824dbfa04949" containerName="console" Jul 07 17:45:37.038708 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.038687 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" Jul 07 17:45:37.041121 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.041090 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-798jq\"" Jul 07 17:45:37.041276 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.041094 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 17:45:37.041850 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.041832 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 17:45:37.048286 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.048260 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc"] Jul 07 17:45:37.062274 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.062240 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f02a61bf-afd0-4b8e-af57-95744d6b4766-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc\" (UID: \"f02a61bf-afd0-4b8e-af57-95744d6b4766\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" Jul 07 17:45:37.062466 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.062286 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv9xv\" (UniqueName: \"kubernetes.io/projected/f02a61bf-afd0-4b8e-af57-95744d6b4766-kube-api-access-hv9xv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc\" (UID: \"f02a61bf-afd0-4b8e-af57-95744d6b4766\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" Jul 07 17:45:37.062466 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.062343 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f02a61bf-afd0-4b8e-af57-95744d6b4766-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc\" (UID: \"f02a61bf-afd0-4b8e-af57-95744d6b4766\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" Jul 07 17:45:37.163472 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.163437 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f02a61bf-afd0-4b8e-af57-95744d6b4766-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc\" (UID: \"f02a61bf-afd0-4b8e-af57-95744d6b4766\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" Jul 07 17:45:37.163472 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.163485 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv9xv\" (UniqueName: \"kubernetes.io/projected/f02a61bf-afd0-4b8e-af57-95744d6b4766-kube-api-access-hv9xv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc\" (UID: \"f02a61bf-afd0-4b8e-af57-95744d6b4766\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" Jul 07 17:45:37.163691 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.163513 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f02a61bf-afd0-4b8e-af57-95744d6b4766-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc\" (UID: \"f02a61bf-afd0-4b8e-af57-95744d6b4766\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" Jul 07 17:45:37.163832 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.163812 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f02a61bf-afd0-4b8e-af57-95744d6b4766-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc\" (UID: \"f02a61bf-afd0-4b8e-af57-95744d6b4766\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" Jul 07 17:45:37.163880 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.163864 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f02a61bf-afd0-4b8e-af57-95744d6b4766-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc\" (UID: \"f02a61bf-afd0-4b8e-af57-95744d6b4766\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" Jul 07 17:45:37.171895 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.171859 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv9xv\" (UniqueName: \"kubernetes.io/projected/f02a61bf-afd0-4b8e-af57-95744d6b4766-kube-api-access-hv9xv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc\" (UID: \"f02a61bf-afd0-4b8e-af57-95744d6b4766\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" Jul 07 17:45:37.348787 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.348687 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" Jul 07 17:45:37.495996 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.495952 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc"] Jul 07 17:45:37.500182 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:45:37.500153 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf02a61bf_afd0_4b8e_af57_95744d6b4766.slice/crio-a3afd6535c7f36e20830f1c05e336f6f63fe3eac353b5cb4fe38b5b364952b6d WatchSource:0}: Error finding container a3afd6535c7f36e20830f1c05e336f6f63fe3eac353b5cb4fe38b5b364952b6d: Status 404 returned error can't find the container with id a3afd6535c7f36e20830f1c05e336f6f63fe3eac353b5cb4fe38b5b364952b6d Jul 07 17:45:37.502178 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.502155 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 17:45:37.968204 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:37.968168 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" event={"ID":"f02a61bf-afd0-4b8e-af57-95744d6b4766","Type":"ContainerStarted","Data":"a3afd6535c7f36e20830f1c05e336f6f63fe3eac353b5cb4fe38b5b364952b6d"} Jul 07 17:45:46.002764 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:46.002726 2577 generic.go:356] "Generic (PLEG): container finished" podID="f02a61bf-afd0-4b8e-af57-95744d6b4766" containerID="4d9ff8f089ecbfbec164b1bc38b12003d1e640918f952ebf4810c08aef96d4fa" exitCode=0 Jul 07 17:45:46.003289 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:46.002810 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" event={"ID":"f02a61bf-afd0-4b8e-af57-95744d6b4766","Type":"ContainerDied","Data":"4d9ff8f089ecbfbec164b1bc38b12003d1e640918f952ebf4810c08aef96d4fa"} Jul 07 17:45:48.013981 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:48.013942 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" event={"ID":"f02a61bf-afd0-4b8e-af57-95744d6b4766","Type":"ContainerStarted","Data":"16fe0659621b4bdef28283383f064fa15ba77528595596ffd5577bbc6e694bf1"} Jul 07 17:45:49.019376 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:49.019335 2577 generic.go:356] "Generic (PLEG): container finished" podID="f02a61bf-afd0-4b8e-af57-95744d6b4766" containerID="16fe0659621b4bdef28283383f064fa15ba77528595596ffd5577bbc6e694bf1" exitCode=0 Jul 07 17:45:49.019783 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:49.019431 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" event={"ID":"f02a61bf-afd0-4b8e-af57-95744d6b4766","Type":"ContainerDied","Data":"16fe0659621b4bdef28283383f064fa15ba77528595596ffd5577bbc6e694bf1"} Jul 07 17:45:59.063784 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:59.063748 2577 generic.go:356] "Generic (PLEG): container finished" podID="f02a61bf-afd0-4b8e-af57-95744d6b4766" containerID="8d5ea4190caa8801ad7d2a315b8c3ac81e5258e0dd2da15ba4c669bcb2633da4" exitCode=0 Jul 07 17:45:59.064231 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:45:59.063833 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" event={"ID":"f02a61bf-afd0-4b8e-af57-95744d6b4766","Type":"ContainerDied","Data":"8d5ea4190caa8801ad7d2a315b8c3ac81e5258e0dd2da15ba4c669bcb2633da4"} Jul 07 17:46:00.200364 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:00.200326 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" Jul 07 17:46:00.259138 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:00.259095 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hv9xv\" (UniqueName: \"kubernetes.io/projected/f02a61bf-afd0-4b8e-af57-95744d6b4766-kube-api-access-hv9xv\") pod \"f02a61bf-afd0-4b8e-af57-95744d6b4766\" (UID: \"f02a61bf-afd0-4b8e-af57-95744d6b4766\") " Jul 07 17:46:00.259330 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:00.259150 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f02a61bf-afd0-4b8e-af57-95744d6b4766-util\") pod \"f02a61bf-afd0-4b8e-af57-95744d6b4766\" (UID: \"f02a61bf-afd0-4b8e-af57-95744d6b4766\") " Jul 07 17:46:00.259330 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:00.259199 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f02a61bf-afd0-4b8e-af57-95744d6b4766-bundle\") pod \"f02a61bf-afd0-4b8e-af57-95744d6b4766\" (UID: \"f02a61bf-afd0-4b8e-af57-95744d6b4766\") " Jul 07 17:46:00.259901 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:00.259869 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f02a61bf-afd0-4b8e-af57-95744d6b4766-bundle" (OuterVolumeSpecName: "bundle") pod "f02a61bf-afd0-4b8e-af57-95744d6b4766" (UID: "f02a61bf-afd0-4b8e-af57-95744d6b4766"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:46:00.261668 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:00.261637 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f02a61bf-afd0-4b8e-af57-95744d6b4766-kube-api-access-hv9xv" (OuterVolumeSpecName: "kube-api-access-hv9xv") pod "f02a61bf-afd0-4b8e-af57-95744d6b4766" (UID: "f02a61bf-afd0-4b8e-af57-95744d6b4766"). InnerVolumeSpecName "kube-api-access-hv9xv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:46:00.264314 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:00.264280 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f02a61bf-afd0-4b8e-af57-95744d6b4766-util" (OuterVolumeSpecName: "util") pod "f02a61bf-afd0-4b8e-af57-95744d6b4766" (UID: "f02a61bf-afd0-4b8e-af57-95744d6b4766"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:46:00.360638 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:00.360548 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv9xv\" (UniqueName: \"kubernetes.io/projected/f02a61bf-afd0-4b8e-af57-95744d6b4766-kube-api-access-hv9xv\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:46:00.360638 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:00.360581 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f02a61bf-afd0-4b8e-af57-95744d6b4766-util\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:46:00.360638 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:00.360592 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f02a61bf-afd0-4b8e-af57-95744d6b4766-bundle\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:46:01.074147 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:01.074103 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" event={"ID":"f02a61bf-afd0-4b8e-af57-95744d6b4766","Type":"ContainerDied","Data":"a3afd6535c7f36e20830f1c05e336f6f63fe3eac353b5cb4fe38b5b364952b6d"} Jul 07 17:46:01.074336 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:01.074171 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a3afd6535c7f36e20830f1c05e336f6f63fe3eac353b5cb4fe38b5b364952b6d" Jul 07 17:46:01.074336 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:01.074129 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sxkhc" Jul 07 17:46:05.007575 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.007537 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fxvp4"] Jul 07 17:46:05.007968 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.007904 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f02a61bf-afd0-4b8e-af57-95744d6b4766" containerName="pull" Jul 07 17:46:05.007968 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.007918 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f02a61bf-afd0-4b8e-af57-95744d6b4766" containerName="pull" Jul 07 17:46:05.007968 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.007925 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f02a61bf-afd0-4b8e-af57-95744d6b4766" containerName="extract" Jul 07 17:46:05.007968 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.007931 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f02a61bf-afd0-4b8e-af57-95744d6b4766" containerName="extract" Jul 07 17:46:05.007968 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.007941 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f02a61bf-afd0-4b8e-af57-95744d6b4766" containerName="util" Jul 07 17:46:05.007968 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.007946 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f02a61bf-afd0-4b8e-af57-95744d6b4766" containerName="util" Jul 07 17:46:05.008172 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.007994 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f02a61bf-afd0-4b8e-af57-95744d6b4766" containerName="extract" Jul 07 17:46:05.010454 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.010430 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fxvp4" Jul 07 17:46:05.013175 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.013142 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:46:05.013330 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.013176 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-nn6s2\"" Jul 07 17:46:05.013330 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.013184 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 17:46:05.027785 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.027749 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fxvp4"] Jul 07 17:46:05.105909 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.105864 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt8w7\" (UniqueName: \"kubernetes.io/projected/7fb8ac64-e2c9-4a09-9ddb-2174d6581014-kube-api-access-wt8w7\") pod \"cert-manager-operator-controller-manager-55c444bf86-fxvp4\" (UID: \"7fb8ac64-e2c9-4a09-9ddb-2174d6581014\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fxvp4" Jul 07 17:46:05.106141 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.105922 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fb8ac64-e2c9-4a09-9ddb-2174d6581014-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fxvp4\" (UID: \"7fb8ac64-e2c9-4a09-9ddb-2174d6581014\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fxvp4" Jul 07 17:46:05.206770 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.206725 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fb8ac64-e2c9-4a09-9ddb-2174d6581014-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fxvp4\" (UID: \"7fb8ac64-e2c9-4a09-9ddb-2174d6581014\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fxvp4" Jul 07 17:46:05.206962 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.206824 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt8w7\" (UniqueName: \"kubernetes.io/projected/7fb8ac64-e2c9-4a09-9ddb-2174d6581014-kube-api-access-wt8w7\") pod \"cert-manager-operator-controller-manager-55c444bf86-fxvp4\" (UID: \"7fb8ac64-e2c9-4a09-9ddb-2174d6581014\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fxvp4" Jul 07 17:46:05.207162 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.207140 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fb8ac64-e2c9-4a09-9ddb-2174d6581014-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fxvp4\" (UID: \"7fb8ac64-e2c9-4a09-9ddb-2174d6581014\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fxvp4" Jul 07 17:46:05.216021 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.215984 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt8w7\" (UniqueName: \"kubernetes.io/projected/7fb8ac64-e2c9-4a09-9ddb-2174d6581014-kube-api-access-wt8w7\") pod \"cert-manager-operator-controller-manager-55c444bf86-fxvp4\" (UID: \"7fb8ac64-e2c9-4a09-9ddb-2174d6581014\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fxvp4" Jul 07 17:46:05.320161 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.320053 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fxvp4" Jul 07 17:46:05.506662 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:05.506628 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fxvp4"] Jul 07 17:46:05.510405 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:46:05.510368 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fb8ac64_e2c9_4a09_9ddb_2174d6581014.slice/crio-ee065a09139bd47846dc4f5d29b276854bc950b4f949bccb1530eb918eafb89b WatchSource:0}: Error finding container ee065a09139bd47846dc4f5d29b276854bc950b4f949bccb1530eb918eafb89b: Status 404 returned error can't find the container with id ee065a09139bd47846dc4f5d29b276854bc950b4f949bccb1530eb918eafb89b Jul 07 17:46:06.095185 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:06.095143 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fxvp4" event={"ID":"7fb8ac64-e2c9-4a09-9ddb-2174d6581014","Type":"ContainerStarted","Data":"ee065a09139bd47846dc4f5d29b276854bc950b4f949bccb1530eb918eafb89b"} Jul 07 17:46:08.106316 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:08.106209 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fxvp4" event={"ID":"7fb8ac64-e2c9-4a09-9ddb-2174d6581014","Type":"ContainerStarted","Data":"48b18ee99251d1a2019b2abc71a30ce0e42076568ba939e2d964f8cd62de261f"} Jul 07 17:46:08.132156 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:08.132091 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fxvp4" podStartSLOduration=1.802582735 podStartE2EDuration="4.132067167s" totalImagesPullingTime="2.329484432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:46:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:46:08.130136993 +0000 UTC m=+372.169463896" watchObservedRunningTime="2026-07-07 17:46:08.132067167 +0000 UTC m=+372.171394061" Jul 07 17:46:09.002297 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.002259 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt"] Jul 07 17:46:09.006120 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.006097 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" Jul 07 17:46:09.010279 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.010247 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 17:46:09.010467 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.010402 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 17:46:09.011086 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.011069 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-798jq\"" Jul 07 17:46:09.017018 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.016989 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt"] Jul 07 17:46:09.142230 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.142185 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt\" (UID: \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" Jul 07 17:46:09.142230 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.142236 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl87p\" (UniqueName: \"kubernetes.io/projected/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-kube-api-access-tl87p\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt\" (UID: \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" Jul 07 17:46:09.142659 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.142294 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt\" (UID: \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" Jul 07 17:46:09.243512 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.243473 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt\" (UID: \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" Jul 07 17:46:09.243691 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.243554 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt\" (UID: \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" Jul 07 17:46:09.243691 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.243583 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl87p\" (UniqueName: \"kubernetes.io/projected/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-kube-api-access-tl87p\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt\" (UID: \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" Jul 07 17:46:09.243884 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.243864 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt\" (UID: \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" Jul 07 17:46:09.243967 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.243944 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt\" (UID: \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" Jul 07 17:46:09.253654 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.253585 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl87p\" (UniqueName: \"kubernetes.io/projected/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-kube-api-access-tl87p\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt\" (UID: \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" Jul 07 17:46:09.316610 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.316569 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" Jul 07 17:46:09.478493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:09.478397 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt"] Jul 07 17:46:09.481176 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:46:09.481144 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd42bf80b_337a_4315_ad6d_9f8f1c5f4edf.slice/crio-2eb33555607bc3687f5753ad4d6ef11d80b64781561720f55714c3953428fd0e WatchSource:0}: Error finding container 2eb33555607bc3687f5753ad4d6ef11d80b64781561720f55714c3953428fd0e: Status 404 returned error can't find the container with id 2eb33555607bc3687f5753ad4d6ef11d80b64781561720f55714c3953428fd0e Jul 07 17:46:10.116875 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:10.116841 2577 generic.go:356] "Generic (PLEG): container finished" podID="d42bf80b-337a-4315-ad6d-9f8f1c5f4edf" containerID="6705c5c723684487f0eb70fa7279913b7019cf03406ca50b1ddd3ff3276b71bf" exitCode=0 Jul 07 17:46:10.116875 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:10.116878 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" event={"ID":"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf","Type":"ContainerDied","Data":"6705c5c723684487f0eb70fa7279913b7019cf03406ca50b1ddd3ff3276b71bf"} Jul 07 17:46:10.117141 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:10.116919 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" event={"ID":"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf","Type":"ContainerStarted","Data":"2eb33555607bc3687f5753ad4d6ef11d80b64781561720f55714c3953428fd0e"} Jul 07 17:46:13.138280 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:13.138241 2577 generic.go:356] "Generic (PLEG): container finished" podID="d42bf80b-337a-4315-ad6d-9f8f1c5f4edf" containerID="7faf858398017bca6929aba40fbb39e8a892b3bbbae771063ba1abeae416b1fe" exitCode=0 Jul 07 17:46:13.138683 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:13.138336 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" event={"ID":"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf","Type":"ContainerDied","Data":"7faf858398017bca6929aba40fbb39e8a892b3bbbae771063ba1abeae416b1fe"} Jul 07 17:46:14.145766 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:14.145724 2577 generic.go:356] "Generic (PLEG): container finished" podID="d42bf80b-337a-4315-ad6d-9f8f1c5f4edf" containerID="32a0de9ea94bd7af970d39248045f5a4b54b02fb795e2f612de65c0b08fd89bd" exitCode=0 Jul 07 17:46:14.146198 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:14.145822 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" event={"ID":"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf","Type":"ContainerDied","Data":"32a0de9ea94bd7af970d39248045f5a4b54b02fb795e2f612de65c0b08fd89bd"} Jul 07 17:46:15.287294 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:15.287251 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" Jul 07 17:46:15.398261 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:15.398222 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tl87p\" (UniqueName: \"kubernetes.io/projected/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-kube-api-access-tl87p\") pod \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\" (UID: \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\") " Jul 07 17:46:15.398261 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:15.398265 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-bundle\") pod \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\" (UID: \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\") " Jul 07 17:46:15.398522 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:15.398293 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-util\") pod \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\" (UID: \"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf\") " Jul 07 17:46:15.398711 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:15.398686 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-bundle" (OuterVolumeSpecName: "bundle") pod "d42bf80b-337a-4315-ad6d-9f8f1c5f4edf" (UID: "d42bf80b-337a-4315-ad6d-9f8f1c5f4edf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:46:15.400656 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:15.400581 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-kube-api-access-tl87p" (OuterVolumeSpecName: "kube-api-access-tl87p") pod "d42bf80b-337a-4315-ad6d-9f8f1c5f4edf" (UID: "d42bf80b-337a-4315-ad6d-9f8f1c5f4edf"). InnerVolumeSpecName "kube-api-access-tl87p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:46:15.403857 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:15.403825 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-util" (OuterVolumeSpecName: "util") pod "d42bf80b-337a-4315-ad6d-9f8f1c5f4edf" (UID: "d42bf80b-337a-4315-ad6d-9f8f1c5f4edf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:46:15.499392 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:15.499347 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl87p\" (UniqueName: \"kubernetes.io/projected/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-kube-api-access-tl87p\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:46:15.499392 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:15.499384 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-bundle\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:46:15.499392 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:15.499393 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d42bf80b-337a-4315-ad6d-9f8f1c5f4edf-util\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:46:16.155994 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:16.155966 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" Jul 07 17:46:16.156268 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:16.155961 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftqwlt" event={"ID":"d42bf80b-337a-4315-ad6d-9f8f1c5f4edf","Type":"ContainerDied","Data":"2eb33555607bc3687f5753ad4d6ef11d80b64781561720f55714c3953428fd0e"} Jul 07 17:46:16.156268 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:16.156095 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2eb33555607bc3687f5753ad4d6ef11d80b64781561720f55714c3953428fd0e" Jul 07 17:46:20.141721 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.141685 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vvzw7"] Jul 07 17:46:20.142155 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.141971 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d42bf80b-337a-4315-ad6d-9f8f1c5f4edf" containerName="util" Jul 07 17:46:20.142155 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.141982 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d42bf80b-337a-4315-ad6d-9f8f1c5f4edf" containerName="util" Jul 07 17:46:20.142155 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.142000 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d42bf80b-337a-4315-ad6d-9f8f1c5f4edf" containerName="pull" Jul 07 17:46:20.142155 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.142006 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d42bf80b-337a-4315-ad6d-9f8f1c5f4edf" containerName="pull" Jul 07 17:46:20.142155 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.142017 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d42bf80b-337a-4315-ad6d-9f8f1c5f4edf" containerName="extract" Jul 07 17:46:20.142155 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.142026 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d42bf80b-337a-4315-ad6d-9f8f1c5f4edf" containerName="extract" Jul 07 17:46:20.142155 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.142111 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d42bf80b-337a-4315-ad6d-9f8f1c5f4edf" containerName="extract" Jul 07 17:46:20.144970 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.144951 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-vvzw7" Jul 07 17:46:20.147361 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.147334 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-2dr2d\"" Jul 07 17:46:20.147361 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.147357 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 17:46:20.147969 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.147949 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 17:46:20.152913 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.152605 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vvzw7"] Jul 07 17:46:20.335486 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.335442 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwswg\" (UniqueName: \"kubernetes.io/projected/75177253-e7f0-4fcb-a50a-7e2b8782b6f1-kube-api-access-rwswg\") pod \"cert-manager-55b4d954d5-vvzw7\" (UID: \"75177253-e7f0-4fcb-a50a-7e2b8782b6f1\") " pod="cert-manager/cert-manager-55b4d954d5-vvzw7" Jul 07 17:46:20.335670 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.335512 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/75177253-e7f0-4fcb-a50a-7e2b8782b6f1-bound-sa-token\") pod \"cert-manager-55b4d954d5-vvzw7\" (UID: \"75177253-e7f0-4fcb-a50a-7e2b8782b6f1\") " pod="cert-manager/cert-manager-55b4d954d5-vvzw7" Jul 07 17:46:20.436302 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.436259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwswg\" (UniqueName: \"kubernetes.io/projected/75177253-e7f0-4fcb-a50a-7e2b8782b6f1-kube-api-access-rwswg\") pod \"cert-manager-55b4d954d5-vvzw7\" (UID: \"75177253-e7f0-4fcb-a50a-7e2b8782b6f1\") " pod="cert-manager/cert-manager-55b4d954d5-vvzw7" Jul 07 17:46:20.436487 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.436337 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/75177253-e7f0-4fcb-a50a-7e2b8782b6f1-bound-sa-token\") pod \"cert-manager-55b4d954d5-vvzw7\" (UID: \"75177253-e7f0-4fcb-a50a-7e2b8782b6f1\") " pod="cert-manager/cert-manager-55b4d954d5-vvzw7" Jul 07 17:46:20.444965 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.444925 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/75177253-e7f0-4fcb-a50a-7e2b8782b6f1-bound-sa-token\") pod \"cert-manager-55b4d954d5-vvzw7\" (UID: \"75177253-e7f0-4fcb-a50a-7e2b8782b6f1\") " pod="cert-manager/cert-manager-55b4d954d5-vvzw7" Jul 07 17:46:20.445265 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.445244 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwswg\" (UniqueName: \"kubernetes.io/projected/75177253-e7f0-4fcb-a50a-7e2b8782b6f1-kube-api-access-rwswg\") pod \"cert-manager-55b4d954d5-vvzw7\" (UID: \"75177253-e7f0-4fcb-a50a-7e2b8782b6f1\") " pod="cert-manager/cert-manager-55b4d954d5-vvzw7" Jul 07 17:46:20.465183 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.465148 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-vvzw7" Jul 07 17:46:20.607658 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:20.607621 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vvzw7"] Jul 07 17:46:20.610852 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:46:20.610822 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75177253_e7f0_4fcb_a50a_7e2b8782b6f1.slice/crio-d71871d6a4bf632447614628d6535ac8709e354792fb0bfe6b29941cadec40f7 WatchSource:0}: Error finding container d71871d6a4bf632447614628d6535ac8709e354792fb0bfe6b29941cadec40f7: Status 404 returned error can't find the container with id d71871d6a4bf632447614628d6535ac8709e354792fb0bfe6b29941cadec40f7 Jul 07 17:46:21.179482 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:21.179441 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-vvzw7" event={"ID":"75177253-e7f0-4fcb-a50a-7e2b8782b6f1","Type":"ContainerStarted","Data":"d71871d6a4bf632447614628d6535ac8709e354792fb0bfe6b29941cadec40f7"} Jul 07 17:46:24.194365 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:24.194320 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-vvzw7" event={"ID":"75177253-e7f0-4fcb-a50a-7e2b8782b6f1","Type":"ContainerStarted","Data":"e65f10df8e31082368d1c5da948bc6dbe35fba5543845d8238d607e4f914a5ff"} Jul 07 17:46:24.211839 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:24.211781 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-vvzw7" podStartSLOduration=1.3140620410000001 podStartE2EDuration="4.211763953s" totalImagesPullingTime="2.897701912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:46:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:46:24.210014776 +0000 UTC m=+388.249341679" watchObservedRunningTime="2026-07-07 17:46:24.211763953 +0000 UTC m=+388.251090856" Jul 07 17:46:28.882744 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:28.882697 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl"] Jul 07 17:46:28.886925 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:28.886893 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" Jul 07 17:46:28.890339 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:28.890309 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 17:46:28.890519 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:28.890485 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 17:46:28.890624 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:28.890411 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-798jq\"" Jul 07 17:46:28.891452 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:28.891425 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl"] Jul 07 17:46:28.902010 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:28.901972 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/026aa086-133c-4c6c-bd71-450dd06471dc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl\" (UID: \"026aa086-133c-4c6c-bd71-450dd06471dc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" Jul 07 17:46:28.902235 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:28.902034 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/026aa086-133c-4c6c-bd71-450dd06471dc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl\" (UID: \"026aa086-133c-4c6c-bd71-450dd06471dc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" Jul 07 17:46:28.902235 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:28.902099 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qddsr\" (UniqueName: \"kubernetes.io/projected/026aa086-133c-4c6c-bd71-450dd06471dc-kube-api-access-qddsr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl\" (UID: \"026aa086-133c-4c6c-bd71-450dd06471dc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" Jul 07 17:46:29.003350 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:29.003306 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/026aa086-133c-4c6c-bd71-450dd06471dc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl\" (UID: \"026aa086-133c-4c6c-bd71-450dd06471dc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" Jul 07 17:46:29.003350 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:29.003348 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/026aa086-133c-4c6c-bd71-450dd06471dc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl\" (UID: \"026aa086-133c-4c6c-bd71-450dd06471dc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" Jul 07 17:46:29.003588 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:29.003371 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qddsr\" (UniqueName: \"kubernetes.io/projected/026aa086-133c-4c6c-bd71-450dd06471dc-kube-api-access-qddsr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl\" (UID: \"026aa086-133c-4c6c-bd71-450dd06471dc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" Jul 07 17:46:29.003822 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:29.003794 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/026aa086-133c-4c6c-bd71-450dd06471dc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl\" (UID: \"026aa086-133c-4c6c-bd71-450dd06471dc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" Jul 07 17:46:29.003862 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:29.003806 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/026aa086-133c-4c6c-bd71-450dd06471dc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl\" (UID: \"026aa086-133c-4c6c-bd71-450dd06471dc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" Jul 07 17:46:29.011733 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:29.011694 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qddsr\" (UniqueName: \"kubernetes.io/projected/026aa086-133c-4c6c-bd71-450dd06471dc-kube-api-access-qddsr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl\" (UID: \"026aa086-133c-4c6c-bd71-450dd06471dc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" Jul 07 17:46:29.197389 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:29.197341 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" Jul 07 17:46:29.341391 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:29.341193 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl"] Jul 07 17:46:29.344349 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:46:29.344317 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026aa086_133c_4c6c_bd71_450dd06471dc.slice/crio-b832ca39353a6304ea14f22f91ceda1c1433656daebbf34dd4404eed850abd28 WatchSource:0}: Error finding container b832ca39353a6304ea14f22f91ceda1c1433656daebbf34dd4404eed850abd28: Status 404 returned error can't find the container with id b832ca39353a6304ea14f22f91ceda1c1433656daebbf34dd4404eed850abd28 Jul 07 17:46:30.222598 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:30.222553 2577 generic.go:356] "Generic (PLEG): container finished" podID="026aa086-133c-4c6c-bd71-450dd06471dc" containerID="00a394fb7031eac752e4d6cfc3f75264ce18402a8a69c7b681441469def5c6f8" exitCode=0 Jul 07 17:46:30.222987 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:30.222594 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" event={"ID":"026aa086-133c-4c6c-bd71-450dd06471dc","Type":"ContainerDied","Data":"00a394fb7031eac752e4d6cfc3f75264ce18402a8a69c7b681441469def5c6f8"} Jul 07 17:46:30.222987 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:30.222649 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" event={"ID":"026aa086-133c-4c6c-bd71-450dd06471dc","Type":"ContainerStarted","Data":"b832ca39353a6304ea14f22f91ceda1c1433656daebbf34dd4404eed850abd28"} Jul 07 17:46:31.229120 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:31.229082 2577 generic.go:356] "Generic (PLEG): container finished" podID="026aa086-133c-4c6c-bd71-450dd06471dc" containerID="0c9717fe896cf2d00a611d34940d5b3ae01066f5b31715d28feb7bda9ef809f5" exitCode=0 Jul 07 17:46:31.229516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:31.229150 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" event={"ID":"026aa086-133c-4c6c-bd71-450dd06471dc","Type":"ContainerDied","Data":"0c9717fe896cf2d00a611d34940d5b3ae01066f5b31715d28feb7bda9ef809f5"} Jul 07 17:46:32.236221 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:32.236189 2577 generic.go:356] "Generic (PLEG): container finished" podID="026aa086-133c-4c6c-bd71-450dd06471dc" containerID="67f576d76780cccc78439354e5abe99a1c390923e417d2bac6a34f0b93e51458" exitCode=0 Jul 07 17:46:32.236625 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:32.236240 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" event={"ID":"026aa086-133c-4c6c-bd71-450dd06471dc","Type":"ContainerDied","Data":"67f576d76780cccc78439354e5abe99a1c390923e417d2bac6a34f0b93e51458"} Jul 07 17:46:33.380912 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:33.380877 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" Jul 07 17:46:33.433811 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:33.433760 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/026aa086-133c-4c6c-bd71-450dd06471dc-util\") pod \"026aa086-133c-4c6c-bd71-450dd06471dc\" (UID: \"026aa086-133c-4c6c-bd71-450dd06471dc\") " Jul 07 17:46:33.433984 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:33.433820 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qddsr\" (UniqueName: \"kubernetes.io/projected/026aa086-133c-4c6c-bd71-450dd06471dc-kube-api-access-qddsr\") pod \"026aa086-133c-4c6c-bd71-450dd06471dc\" (UID: \"026aa086-133c-4c6c-bd71-450dd06471dc\") " Jul 07 17:46:33.433984 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:33.433850 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/026aa086-133c-4c6c-bd71-450dd06471dc-bundle\") pod \"026aa086-133c-4c6c-bd71-450dd06471dc\" (UID: \"026aa086-133c-4c6c-bd71-450dd06471dc\") " Jul 07 17:46:33.434833 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:33.434794 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/026aa086-133c-4c6c-bd71-450dd06471dc-bundle" (OuterVolumeSpecName: "bundle") pod "026aa086-133c-4c6c-bd71-450dd06471dc" (UID: "026aa086-133c-4c6c-bd71-450dd06471dc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:46:33.436188 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:33.436150 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/026aa086-133c-4c6c-bd71-450dd06471dc-kube-api-access-qddsr" (OuterVolumeSpecName: "kube-api-access-qddsr") pod "026aa086-133c-4c6c-bd71-450dd06471dc" (UID: "026aa086-133c-4c6c-bd71-450dd06471dc"). InnerVolumeSpecName "kube-api-access-qddsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:46:33.440324 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:33.440296 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/026aa086-133c-4c6c-bd71-450dd06471dc-util" (OuterVolumeSpecName: "util") pod "026aa086-133c-4c6c-bd71-450dd06471dc" (UID: "026aa086-133c-4c6c-bd71-450dd06471dc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:46:33.535105 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:33.534988 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/026aa086-133c-4c6c-bd71-450dd06471dc-util\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:46:33.535105 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:33.535030 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qddsr\" (UniqueName: \"kubernetes.io/projected/026aa086-133c-4c6c-bd71-450dd06471dc-kube-api-access-qddsr\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:46:33.535105 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:33.535064 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/026aa086-133c-4c6c-bd71-450dd06471dc-bundle\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:46:34.247832 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:34.247798 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" Jul 07 17:46:34.248025 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:34.247798 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tkkl" event={"ID":"026aa086-133c-4c6c-bd71-450dd06471dc","Type":"ContainerDied","Data":"b832ca39353a6304ea14f22f91ceda1c1433656daebbf34dd4404eed850abd28"} Jul 07 17:46:34.248025 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:34.247929 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b832ca39353a6304ea14f22f91ceda1c1433656daebbf34dd4404eed850abd28" Jul 07 17:46:46.159680 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.159640 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j"] Jul 07 17:46:46.160163 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.159939 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="026aa086-133c-4c6c-bd71-450dd06471dc" containerName="util" Jul 07 17:46:46.160163 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.159950 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="026aa086-133c-4c6c-bd71-450dd06471dc" containerName="util" Jul 07 17:46:46.160163 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.159959 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="026aa086-133c-4c6c-bd71-450dd06471dc" containerName="pull" Jul 07 17:46:46.160163 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.159965 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="026aa086-133c-4c6c-bd71-450dd06471dc" containerName="pull" Jul 07 17:46:46.160163 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.159973 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="026aa086-133c-4c6c-bd71-450dd06471dc" containerName="extract" Jul 07 17:46:46.160163 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.159978 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="026aa086-133c-4c6c-bd71-450dd06471dc" containerName="extract" Jul 07 17:46:46.160163 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.160031 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="026aa086-133c-4c6c-bd71-450dd06471dc" containerName="extract" Jul 07 17:46:46.163384 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.163362 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.166457 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.166423 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 17:46:46.166608 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.166509 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 17:46:46.166608 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.166582 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 17:46:46.166718 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.166658 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-727k9\"" Jul 07 17:46:46.166821 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.166808 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 17:46:46.182183 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.182151 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j"] Jul 07 17:46:46.225474 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.225434 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e53c694f-4f4f-4c73-8155-e3681092329d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.225474 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.225483 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9lkn\" (UniqueName: \"kubernetes.io/projected/e53c694f-4f4f-4c73-8155-e3681092329d-kube-api-access-h9lkn\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.225717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.225508 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e53c694f-4f4f-4c73-8155-e3681092329d-manifests\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.225717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.225574 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e53c694f-4f4f-4c73-8155-e3681092329d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.225717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.225628 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e53c694f-4f4f-4c73-8155-e3681092329d-tmp\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.326662 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.326621 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e53c694f-4f4f-4c73-8155-e3681092329d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.326847 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.326672 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9lkn\" (UniqueName: \"kubernetes.io/projected/e53c694f-4f4f-4c73-8155-e3681092329d-kube-api-access-h9lkn\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.326847 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.326810 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e53c694f-4f4f-4c73-8155-e3681092329d-manifests\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.326985 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.326894 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e53c694f-4f4f-4c73-8155-e3681092329d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.326985 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.326974 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e53c694f-4f4f-4c73-8155-e3681092329d-tmp\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.327317 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.327287 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e53c694f-4f4f-4c73-8155-e3681092329d-manifests\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.329298 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.329270 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e53c694f-4f4f-4c73-8155-e3681092329d-tmp\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.329570 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.329545 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e53c694f-4f4f-4c73-8155-e3681092329d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.329689 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.329674 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e53c694f-4f4f-4c73-8155-e3681092329d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.346086 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.346026 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9lkn\" (UniqueName: \"kubernetes.io/projected/e53c694f-4f4f-4c73-8155-e3681092329d-kube-api-access-h9lkn\") pod \"opendatahub-operator-controller-manager-6f77d785b9-ffm2j\" (UID: \"e53c694f-4f4f-4c73-8155-e3681092329d\") " pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.474920 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.474865 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:46.644573 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:46.644541 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j"] Jul 07 17:46:46.648742 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:46:46.648716 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode53c694f_4f4f_4c73_8155_e3681092329d.slice/crio-151dc9ab03064a6569d655a221a4bc538cd5529f0c84c9edb9caabdf152b7a0d WatchSource:0}: Error finding container 151dc9ab03064a6569d655a221a4bc538cd5529f0c84c9edb9caabdf152b7a0d: Status 404 returned error can't find the container with id 151dc9ab03064a6569d655a221a4bc538cd5529f0c84c9edb9caabdf152b7a0d Jul 07 17:46:47.313151 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:47.313106 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" event={"ID":"e53c694f-4f4f-4c73-8155-e3681092329d","Type":"ContainerStarted","Data":"151dc9ab03064a6569d655a221a4bc538cd5529f0c84c9edb9caabdf152b7a0d"} Jul 07 17:46:50.332873 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:50.332834 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" event={"ID":"e53c694f-4f4f-4c73-8155-e3681092329d","Type":"ContainerStarted","Data":"079eaa69345dedf37a6c858d08b2abafebce4bbc8d026f6c291f818f23d0813c"} Jul 07 17:46:51.339527 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:51.339487 2577 generic.go:356] "Generic (PLEG): container finished" podID="e53c694f-4f4f-4c73-8155-e3681092329d" containerID="079eaa69345dedf37a6c858d08b2abafebce4bbc8d026f6c291f818f23d0813c" exitCode=0 Jul 07 17:46:51.339527 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:51.339529 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" event={"ID":"e53c694f-4f4f-4c73-8155-e3681092329d","Type":"ContainerDied","Data":"079eaa69345dedf37a6c858d08b2abafebce4bbc8d026f6c291f818f23d0813c"} Jul 07 17:46:52.346372 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:52.346325 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" event={"ID":"e53c694f-4f4f-4c73-8155-e3681092329d","Type":"ContainerStarted","Data":"294c3c292a21a675bf3d2e3ecd14b8215c541960994505bed042af1cad6b2ff9"} Jul 07 17:46:52.346837 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:52.346421 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:46:52.372144 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:46:52.372082 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" podStartSLOduration=2.086054781 podStartE2EDuration="6.372062079s" totalImagesPullingTime="3.226094298s" totalInitContainerRuntime="1.059913s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:46:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:46:52.370097058 +0000 UTC m=+416.409423955" watchObservedRunningTime="2026-07-07 17:46:52.372062079 +0000 UTC m=+416.411389057" Jul 07 17:47:03.353717 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.353685 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f77d785b9-ffm2j" Jul 07 17:47:03.805106 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.805066 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6"] Jul 07 17:47:03.814630 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.814596 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:03.817922 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.817885 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 17:47:03.818145 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.817949 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 17:47:03.818145 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.817889 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bxmk4\"" Jul 07 17:47:03.818145 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.817998 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 17:47:03.818145 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.817950 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 17:47:03.818145 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.817889 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 17:47:03.819987 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.819958 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6"] Jul 07 17:47:03.884222 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.884185 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41d49a3e-d7ca-434d-9f37-6a310bdece6b-cert\") pod \"lws-controller-manager-7789f458f-fzlt6\" (UID: \"41d49a3e-d7ca-434d-9f37-6a310bdece6b\") " pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:03.884434 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.884242 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/41d49a3e-d7ca-434d-9f37-6a310bdece6b-metrics-cert\") pod \"lws-controller-manager-7789f458f-fzlt6\" (UID: \"41d49a3e-d7ca-434d-9f37-6a310bdece6b\") " pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:03.884434 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.884339 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/41d49a3e-d7ca-434d-9f37-6a310bdece6b-manager-config\") pod \"lws-controller-manager-7789f458f-fzlt6\" (UID: \"41d49a3e-d7ca-434d-9f37-6a310bdece6b\") " pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:03.884434 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.884403 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lc6q\" (UniqueName: \"kubernetes.io/projected/41d49a3e-d7ca-434d-9f37-6a310bdece6b-kube-api-access-6lc6q\") pod \"lws-controller-manager-7789f458f-fzlt6\" (UID: \"41d49a3e-d7ca-434d-9f37-6a310bdece6b\") " pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:03.985271 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.985226 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/41d49a3e-d7ca-434d-9f37-6a310bdece6b-metrics-cert\") pod \"lws-controller-manager-7789f458f-fzlt6\" (UID: \"41d49a3e-d7ca-434d-9f37-6a310bdece6b\") " pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:03.985271 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.985278 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/41d49a3e-d7ca-434d-9f37-6a310bdece6b-manager-config\") pod \"lws-controller-manager-7789f458f-fzlt6\" (UID: \"41d49a3e-d7ca-434d-9f37-6a310bdece6b\") " pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:03.985491 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.985318 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lc6q\" (UniqueName: \"kubernetes.io/projected/41d49a3e-d7ca-434d-9f37-6a310bdece6b-kube-api-access-6lc6q\") pod \"lws-controller-manager-7789f458f-fzlt6\" (UID: \"41d49a3e-d7ca-434d-9f37-6a310bdece6b\") " pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:03.985491 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.985347 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41d49a3e-d7ca-434d-9f37-6a310bdece6b-cert\") pod \"lws-controller-manager-7789f458f-fzlt6\" (UID: \"41d49a3e-d7ca-434d-9f37-6a310bdece6b\") " pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:03.986106 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.986077 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/41d49a3e-d7ca-434d-9f37-6a310bdece6b-manager-config\") pod \"lws-controller-manager-7789f458f-fzlt6\" (UID: \"41d49a3e-d7ca-434d-9f37-6a310bdece6b\") " pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:03.987931 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.987903 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41d49a3e-d7ca-434d-9f37-6a310bdece6b-cert\") pod \"lws-controller-manager-7789f458f-fzlt6\" (UID: \"41d49a3e-d7ca-434d-9f37-6a310bdece6b\") " pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:03.988100 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.988078 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/41d49a3e-d7ca-434d-9f37-6a310bdece6b-metrics-cert\") pod \"lws-controller-manager-7789f458f-fzlt6\" (UID: \"41d49a3e-d7ca-434d-9f37-6a310bdece6b\") " pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:03.995246 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:03.995206 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lc6q\" (UniqueName: \"kubernetes.io/projected/41d49a3e-d7ca-434d-9f37-6a310bdece6b-kube-api-access-6lc6q\") pod \"lws-controller-manager-7789f458f-fzlt6\" (UID: \"41d49a3e-d7ca-434d-9f37-6a310bdece6b\") " pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:04.125750 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:04.125644 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:04.277342 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:04.277299 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6"] Jul 07 17:47:04.281317 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:47:04.281273 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41d49a3e_d7ca_434d_9f37_6a310bdece6b.slice/crio-56f9aad5ff3393fae2df762c3888b3b166726d2adabf6f791c1e289f5d23aa86 WatchSource:0}: Error finding container 56f9aad5ff3393fae2df762c3888b3b166726d2adabf6f791c1e289f5d23aa86: Status 404 returned error can't find the container with id 56f9aad5ff3393fae2df762c3888b3b166726d2adabf6f791c1e289f5d23aa86 Jul 07 17:47:04.411460 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:04.411360 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" event={"ID":"41d49a3e-d7ca-434d-9f37-6a310bdece6b","Type":"ContainerStarted","Data":"56f9aad5ff3393fae2df762c3888b3b166726d2adabf6f791c1e289f5d23aa86"} Jul 07 17:47:06.425280 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.425225 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" event={"ID":"41d49a3e-d7ca-434d-9f37-6a310bdece6b","Type":"ContainerStarted","Data":"2841fcf48e2c53c1d468015e516cb530225be18f4b15c22a0be4d49d4380e169"} Jul 07 17:47:06.425746 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.425537 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:47:06.663311 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.663243 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" podStartSLOduration=2.332999963 podStartE2EDuration="3.663217583s" totalImagesPullingTime="1.33021762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:47:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:47:06.459570506 +0000 UTC m=+430.498897409" watchObservedRunningTime="2026-07-07 17:47:06.663217583 +0000 UTC m=+430.702544487" Jul 07 17:47:06.664961 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.664934 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8"] Jul 07 17:47:06.668268 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.668242 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8" Jul 07 17:47:06.671524 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.671500 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 17:47:06.671685 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.671667 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-99jl5\"" Jul 07 17:47:06.672230 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.672214 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 17:47:06.695500 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.695465 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8"] Jul 07 17:47:06.811959 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.811915 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27rwk\" (UniqueName: \"kubernetes.io/projected/bd6fd06e-fe89-4331-9b94-409899ae1ce7-kube-api-access-27rwk\") pod \"kube-auth-proxy-5675bf5659-wf8n8\" (UID: \"bd6fd06e-fe89-4331-9b94-409899ae1ce7\") " pod="openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8" Jul 07 17:47:06.811959 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.811963 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd6fd06e-fe89-4331-9b94-409899ae1ce7-tmp\") pod \"kube-auth-proxy-5675bf5659-wf8n8\" (UID: \"bd6fd06e-fe89-4331-9b94-409899ae1ce7\") " pod="openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8" Jul 07 17:47:06.812230 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.812078 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd6fd06e-fe89-4331-9b94-409899ae1ce7-tls-certs\") pod \"kube-auth-proxy-5675bf5659-wf8n8\" (UID: \"bd6fd06e-fe89-4331-9b94-409899ae1ce7\") " pod="openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8" Jul 07 17:47:06.913560 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.913512 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd6fd06e-fe89-4331-9b94-409899ae1ce7-tls-certs\") pod \"kube-auth-proxy-5675bf5659-wf8n8\" (UID: \"bd6fd06e-fe89-4331-9b94-409899ae1ce7\") " pod="openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8" Jul 07 17:47:06.913784 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.913614 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27rwk\" (UniqueName: \"kubernetes.io/projected/bd6fd06e-fe89-4331-9b94-409899ae1ce7-kube-api-access-27rwk\") pod \"kube-auth-proxy-5675bf5659-wf8n8\" (UID: \"bd6fd06e-fe89-4331-9b94-409899ae1ce7\") " pod="openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8" Jul 07 17:47:06.913784 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.913635 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd6fd06e-fe89-4331-9b94-409899ae1ce7-tmp\") pod \"kube-auth-proxy-5675bf5659-wf8n8\" (UID: \"bd6fd06e-fe89-4331-9b94-409899ae1ce7\") " pod="openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8" Jul 07 17:47:06.916181 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.916150 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd6fd06e-fe89-4331-9b94-409899ae1ce7-tmp\") pod \"kube-auth-proxy-5675bf5659-wf8n8\" (UID: \"bd6fd06e-fe89-4331-9b94-409899ae1ce7\") " pod="openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8" Jul 07 17:47:06.916372 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.916355 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd6fd06e-fe89-4331-9b94-409899ae1ce7-tls-certs\") pod \"kube-auth-proxy-5675bf5659-wf8n8\" (UID: \"bd6fd06e-fe89-4331-9b94-409899ae1ce7\") " pod="openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8" Jul 07 17:47:06.930313 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.930282 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27rwk\" (UniqueName: \"kubernetes.io/projected/bd6fd06e-fe89-4331-9b94-409899ae1ce7-kube-api-access-27rwk\") pod \"kube-auth-proxy-5675bf5659-wf8n8\" (UID: \"bd6fd06e-fe89-4331-9b94-409899ae1ce7\") " pod="openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8" Jul 07 17:47:06.979271 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:06.979236 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8" Jul 07 17:47:07.191159 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:07.191121 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8"] Jul 07 17:47:07.191853 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:47:07.191825 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd6fd06e_fe89_4331_9b94_409899ae1ce7.slice/crio-e002e5775ef737f8719bfccfe209b668368f3434b4aae314f8b0469905e316ac WatchSource:0}: Error finding container e002e5775ef737f8719bfccfe209b668368f3434b4aae314f8b0469905e316ac: Status 404 returned error can't find the container with id e002e5775ef737f8719bfccfe209b668368f3434b4aae314f8b0469905e316ac Jul 07 17:47:07.431502 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:07.431464 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8" event={"ID":"bd6fd06e-fe89-4331-9b94-409899ae1ce7","Type":"ContainerStarted","Data":"e002e5775ef737f8719bfccfe209b668368f3434b4aae314f8b0469905e316ac"} Jul 07 17:47:11.456521 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:11.456480 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8" event={"ID":"bd6fd06e-fe89-4331-9b94-409899ae1ce7","Type":"ContainerStarted","Data":"32a4c54c6c6bb6b8a083496b3346c7a651b2201948a33f3c895c555a4aeb638d"} Jul 07 17:47:11.487915 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:11.487858 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5675bf5659-wf8n8" podStartSLOduration=2.169063162 podStartE2EDuration="5.487842435s" totalImagesPullingTime="3.318779273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:47:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:47:11.486122071 +0000 UTC m=+435.525448973" watchObservedRunningTime="2026-07-07 17:47:11.487842435 +0000 UTC m=+435.527169346" Jul 07 17:47:17.434640 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:47:17.434556 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7789f458f-fzlt6" Jul 07 17:48:23.005509 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.005471 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq"] Jul 07 17:48:23.008983 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.008959 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" Jul 07 17:48:23.011849 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.011820 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 17:48:23.012023 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.011820 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ptkw2\"" Jul 07 17:48:23.012314 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.012295 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 17:48:23.020710 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.020683 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq"] Jul 07 17:48:23.167523 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.167487 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2rqp\" (UniqueName: \"kubernetes.io/projected/794029e6-db7c-4afd-b417-e66d0fdeb1df-kube-api-access-z2rqp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq\" (UID: \"794029e6-db7c-4afd-b417-e66d0fdeb1df\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" Jul 07 17:48:23.167523 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.167530 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/794029e6-db7c-4afd-b417-e66d0fdeb1df-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq\" (UID: \"794029e6-db7c-4afd-b417-e66d0fdeb1df\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" Jul 07 17:48:23.167755 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.167563 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/794029e6-db7c-4afd-b417-e66d0fdeb1df-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq\" (UID: \"794029e6-db7c-4afd-b417-e66d0fdeb1df\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" Jul 07 17:48:23.268672 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.268581 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2rqp\" (UniqueName: \"kubernetes.io/projected/794029e6-db7c-4afd-b417-e66d0fdeb1df-kube-api-access-z2rqp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq\" (UID: \"794029e6-db7c-4afd-b417-e66d0fdeb1df\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" Jul 07 17:48:23.268672 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.268625 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/794029e6-db7c-4afd-b417-e66d0fdeb1df-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq\" (UID: \"794029e6-db7c-4afd-b417-e66d0fdeb1df\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" Jul 07 17:48:23.268672 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.268670 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/794029e6-db7c-4afd-b417-e66d0fdeb1df-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq\" (UID: \"794029e6-db7c-4afd-b417-e66d0fdeb1df\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" Jul 07 17:48:23.269133 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.269113 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/794029e6-db7c-4afd-b417-e66d0fdeb1df-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq\" (UID: \"794029e6-db7c-4afd-b417-e66d0fdeb1df\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" Jul 07 17:48:23.269218 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.269151 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/794029e6-db7c-4afd-b417-e66d0fdeb1df-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq\" (UID: \"794029e6-db7c-4afd-b417-e66d0fdeb1df\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" Jul 07 17:48:23.280854 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.280808 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2rqp\" (UniqueName: \"kubernetes.io/projected/794029e6-db7c-4afd-b417-e66d0fdeb1df-kube-api-access-z2rqp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq\" (UID: \"794029e6-db7c-4afd-b417-e66d0fdeb1df\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" Jul 07 17:48:23.318592 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.318554 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" Jul 07 17:48:23.376840 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.376801 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs"] Jul 07 17:48:23.384427 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.384396 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" Jul 07 17:48:23.395765 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.395735 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs"] Jul 07 17:48:23.473303 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.473268 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq"] Jul 07 17:48:23.475837 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:48:23.475805 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794029e6_db7c_4afd_b417_e66d0fdeb1df.slice/crio-983741134db47ab1acfca35208c48a325eea0ae19f22cee43206c3ff06373e58 WatchSource:0}: Error finding container 983741134db47ab1acfca35208c48a325eea0ae19f22cee43206c3ff06373e58: Status 404 returned error can't find the container with id 983741134db47ab1acfca35208c48a325eea0ae19f22cee43206c3ff06373e58 Jul 07 17:48:23.572060 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.571994 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62vkk\" (UniqueName: \"kubernetes.io/projected/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-kube-api-access-62vkk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs\" (UID: \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" Jul 07 17:48:23.572244 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.572145 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs\" (UID: \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" Jul 07 17:48:23.572288 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.572255 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs\" (UID: \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" Jul 07 17:48:23.673659 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.673618 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs\" (UID: \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" Jul 07 17:48:23.673866 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.673709 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62vkk\" (UniqueName: \"kubernetes.io/projected/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-kube-api-access-62vkk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs\" (UID: \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" Jul 07 17:48:23.673866 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.673739 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs\" (UID: \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" Jul 07 17:48:23.674082 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.674024 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs\" (UID: \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" Jul 07 17:48:23.674131 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.674087 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs\" (UID: \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" Jul 07 17:48:23.682503 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.682479 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62vkk\" (UniqueName: \"kubernetes.io/projected/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-kube-api-access-62vkk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs\" (UID: \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" Jul 07 17:48:23.702057 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.702013 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" Jul 07 17:48:23.747216 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.747181 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76"] Jul 07 17:48:23.750884 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.750862 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" Jul 07 17:48:23.762517 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.762481 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76"] Jul 07 17:48:23.792849 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.792754 2577 generic.go:356] "Generic (PLEG): container finished" podID="794029e6-db7c-4afd-b417-e66d0fdeb1df" containerID="fa5b42ffa38c21e91904b32483df126c0f4bfa95fcc20e969e2fd7c1927b3757" exitCode=0 Jul 07 17:48:23.792849 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.792808 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" event={"ID":"794029e6-db7c-4afd-b417-e66d0fdeb1df","Type":"ContainerDied","Data":"fa5b42ffa38c21e91904b32483df126c0f4bfa95fcc20e969e2fd7c1927b3757"} Jul 07 17:48:23.793052 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.792869 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" event={"ID":"794029e6-db7c-4afd-b417-e66d0fdeb1df","Type":"ContainerStarted","Data":"983741134db47ab1acfca35208c48a325eea0ae19f22cee43206c3ff06373e58"} Jul 07 17:48:23.874922 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.874838 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c00d28e6-3bfc-4192-b25f-52868d7a2194-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76\" (UID: \"c00d28e6-3bfc-4192-b25f-52868d7a2194\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" Jul 07 17:48:23.875141 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.874959 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c00d28e6-3bfc-4192-b25f-52868d7a2194-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76\" (UID: \"c00d28e6-3bfc-4192-b25f-52868d7a2194\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" Jul 07 17:48:23.875141 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.875004 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrs86\" (UniqueName: \"kubernetes.io/projected/c00d28e6-3bfc-4192-b25f-52868d7a2194-kube-api-access-nrs86\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76\" (UID: \"c00d28e6-3bfc-4192-b25f-52868d7a2194\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" Jul 07 17:48:23.876724 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.876647 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs"] Jul 07 17:48:23.878963 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:48:23.878934 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e7a8d5_d250_44cc_8187_b654dc0f4f3b.slice/crio-d175dc884e4f68cc446d4e9447487ac8148df1967fdcee45c454e372c555f82a WatchSource:0}: Error finding container d175dc884e4f68cc446d4e9447487ac8148df1967fdcee45c454e372c555f82a: Status 404 returned error can't find the container with id d175dc884e4f68cc446d4e9447487ac8148df1967fdcee45c454e372c555f82a Jul 07 17:48:23.975762 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.975728 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c00d28e6-3bfc-4192-b25f-52868d7a2194-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76\" (UID: \"c00d28e6-3bfc-4192-b25f-52868d7a2194\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" Jul 07 17:48:23.975891 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.975803 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrs86\" (UniqueName: \"kubernetes.io/projected/c00d28e6-3bfc-4192-b25f-52868d7a2194-kube-api-access-nrs86\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76\" (UID: \"c00d28e6-3bfc-4192-b25f-52868d7a2194\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" Jul 07 17:48:23.975958 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.975930 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c00d28e6-3bfc-4192-b25f-52868d7a2194-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76\" (UID: \"c00d28e6-3bfc-4192-b25f-52868d7a2194\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" Jul 07 17:48:23.976205 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.976178 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c00d28e6-3bfc-4192-b25f-52868d7a2194-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76\" (UID: \"c00d28e6-3bfc-4192-b25f-52868d7a2194\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" Jul 07 17:48:23.976303 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.976286 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c00d28e6-3bfc-4192-b25f-52868d7a2194-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76\" (UID: \"c00d28e6-3bfc-4192-b25f-52868d7a2194\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" Jul 07 17:48:23.985821 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:23.985777 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrs86\" (UniqueName: \"kubernetes.io/projected/c00d28e6-3bfc-4192-b25f-52868d7a2194-kube-api-access-nrs86\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76\" (UID: \"c00d28e6-3bfc-4192-b25f-52868d7a2194\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" Jul 07 17:48:24.064463 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.064368 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" Jul 07 17:48:24.161673 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.161638 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76"] Jul 07 17:48:24.166084 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.166052 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" Jul 07 17:48:24.177339 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.177185 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76\" (UID: \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" Jul 07 17:48:24.177339 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.177261 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rl4c\" (UniqueName: \"kubernetes.io/projected/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-kube-api-access-4rl4c\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76\" (UID: \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" Jul 07 17:48:24.177339 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.177303 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76\" (UID: \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" Jul 07 17:48:24.177848 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.177825 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76"] Jul 07 17:48:24.214157 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.214133 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76"] Jul 07 17:48:24.216758 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:48:24.216723 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00d28e6_3bfc_4192_b25f_52868d7a2194.slice/crio-ad45f10249af12db9b3a624058555f9ab2c200858b47da8cb24b36545baf4088 WatchSource:0}: Error finding container ad45f10249af12db9b3a624058555f9ab2c200858b47da8cb24b36545baf4088: Status 404 returned error can't find the container with id ad45f10249af12db9b3a624058555f9ab2c200858b47da8cb24b36545baf4088 Jul 07 17:48:24.277795 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.277760 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76\" (UID: \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" Jul 07 17:48:24.277936 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.277830 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rl4c\" (UniqueName: \"kubernetes.io/projected/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-kube-api-access-4rl4c\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76\" (UID: \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" Jul 07 17:48:24.277936 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.277865 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76\" (UID: \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" Jul 07 17:48:24.278259 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.278230 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76\" (UID: \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" Jul 07 17:48:24.278333 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.278233 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76\" (UID: \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" Jul 07 17:48:24.290166 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.290130 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rl4c\" (UniqueName: \"kubernetes.io/projected/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-kube-api-access-4rl4c\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76\" (UID: \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" Jul 07 17:48:24.479644 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.479604 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" Jul 07 17:48:24.676001 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.675844 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76"] Jul 07 17:48:24.679201 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:48:24.679167 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb06fb3cc_a5c4_4386_b5f1_18c1c2b9142d.slice/crio-13e1eae5746ccc2cb5ed19c4acac588c501fffbdc7d3b6fb1f98c279ce941eb5 WatchSource:0}: Error finding container 13e1eae5746ccc2cb5ed19c4acac588c501fffbdc7d3b6fb1f98c279ce941eb5: Status 404 returned error can't find the container with id 13e1eae5746ccc2cb5ed19c4acac588c501fffbdc7d3b6fb1f98c279ce941eb5 Jul 07 17:48:24.799070 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.799003 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" event={"ID":"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d","Type":"ContainerStarted","Data":"945c8555bcfb5543156e9fcd4e1dc7eff335d3c339214a31c3b34147d53c5f97"} Jul 07 17:48:24.799274 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.799080 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" event={"ID":"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d","Type":"ContainerStarted","Data":"13e1eae5746ccc2cb5ed19c4acac588c501fffbdc7d3b6fb1f98c279ce941eb5"} Jul 07 17:48:24.800635 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.800606 2577 generic.go:356] "Generic (PLEG): container finished" podID="51e7a8d5-d250-44cc-8187-b654dc0f4f3b" containerID="0440d422729ca98460f0c6d97e67182d41974f261b5288f3a3da70e8d5467231" exitCode=0 Jul 07 17:48:24.800787 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.800696 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" event={"ID":"51e7a8d5-d250-44cc-8187-b654dc0f4f3b","Type":"ContainerDied","Data":"0440d422729ca98460f0c6d97e67182d41974f261b5288f3a3da70e8d5467231"} Jul 07 17:48:24.800787 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.800753 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" event={"ID":"51e7a8d5-d250-44cc-8187-b654dc0f4f3b","Type":"ContainerStarted","Data":"d175dc884e4f68cc446d4e9447487ac8148df1967fdcee45c454e372c555f82a"} Jul 07 17:48:24.802964 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.802928 2577 generic.go:356] "Generic (PLEG): container finished" podID="794029e6-db7c-4afd-b417-e66d0fdeb1df" containerID="4c9e81e2218322c0e0fa78ea79a0700b1add65cc1556085b67724421aeec0a00" exitCode=0 Jul 07 17:48:24.803105 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.802956 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" event={"ID":"794029e6-db7c-4afd-b417-e66d0fdeb1df","Type":"ContainerDied","Data":"4c9e81e2218322c0e0fa78ea79a0700b1add65cc1556085b67724421aeec0a00"} Jul 07 17:48:24.805077 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.804971 2577 generic.go:356] "Generic (PLEG): container finished" podID="c00d28e6-3bfc-4192-b25f-52868d7a2194" containerID="ececa8761f0b9724ff08443b020cc1b6d3455c04440a120465572bebb3252caf" exitCode=0 Jul 07 17:48:24.805077 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.805011 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" event={"ID":"c00d28e6-3bfc-4192-b25f-52868d7a2194","Type":"ContainerDied","Data":"ececa8761f0b9724ff08443b020cc1b6d3455c04440a120465572bebb3252caf"} Jul 07 17:48:24.805223 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:24.805082 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" event={"ID":"c00d28e6-3bfc-4192-b25f-52868d7a2194","Type":"ContainerStarted","Data":"ad45f10249af12db9b3a624058555f9ab2c200858b47da8cb24b36545baf4088"} Jul 07 17:48:25.811913 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:25.811875 2577 generic.go:356] "Generic (PLEG): container finished" podID="c00d28e6-3bfc-4192-b25f-52868d7a2194" containerID="5d2c6b429611c4cbd372171b18802190fe4eaebf47ebc75a3189a0cb8a88282e" exitCode=0 Jul 07 17:48:25.812424 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:25.811967 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" event={"ID":"c00d28e6-3bfc-4192-b25f-52868d7a2194","Type":"ContainerDied","Data":"5d2c6b429611c4cbd372171b18802190fe4eaebf47ebc75a3189a0cb8a88282e"} Jul 07 17:48:25.813968 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:25.813839 2577 generic.go:356] "Generic (PLEG): container finished" podID="b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d" containerID="945c8555bcfb5543156e9fcd4e1dc7eff335d3c339214a31c3b34147d53c5f97" exitCode=0 Jul 07 17:48:25.813968 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:25.813882 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" event={"ID":"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d","Type":"ContainerDied","Data":"945c8555bcfb5543156e9fcd4e1dc7eff335d3c339214a31c3b34147d53c5f97"} Jul 07 17:48:25.818742 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:25.818696 2577 generic.go:356] "Generic (PLEG): container finished" podID="51e7a8d5-d250-44cc-8187-b654dc0f4f3b" containerID="f66669c447d3cc3ca5fde254c0653ffab920c04a0a57712293c0a5e888a6902b" exitCode=0 Jul 07 17:48:25.818861 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:25.818839 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" event={"ID":"51e7a8d5-d250-44cc-8187-b654dc0f4f3b","Type":"ContainerDied","Data":"f66669c447d3cc3ca5fde254c0653ffab920c04a0a57712293c0a5e888a6902b"} Jul 07 17:48:25.823812 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:25.823752 2577 generic.go:356] "Generic (PLEG): container finished" podID="794029e6-db7c-4afd-b417-e66d0fdeb1df" containerID="0e8a11afbdc1eb75285cfce2c3d87fd350b909b133e33c973612274055d2eb80" exitCode=0 Jul 07 17:48:25.823866 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:25.823816 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" event={"ID":"794029e6-db7c-4afd-b417-e66d0fdeb1df","Type":"ContainerDied","Data":"0e8a11afbdc1eb75285cfce2c3d87fd350b909b133e33c973612274055d2eb80"} Jul 07 17:48:26.831900 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:26.831803 2577 generic.go:356] "Generic (PLEG): container finished" podID="c00d28e6-3bfc-4192-b25f-52868d7a2194" containerID="eaae1a676af7805e5cd493be2c8fd464fec4e0468ed2640ea94c22d748b1f017" exitCode=0 Jul 07 17:48:26.831900 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:26.831881 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" event={"ID":"c00d28e6-3bfc-4192-b25f-52868d7a2194","Type":"ContainerDied","Data":"eaae1a676af7805e5cd493be2c8fd464fec4e0468ed2640ea94c22d748b1f017"} Jul 07 17:48:26.833830 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:26.833795 2577 generic.go:356] "Generic (PLEG): container finished" podID="b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d" containerID="5e30484140cb2c26f5669209626cc2f04d91d0739a1680e5cd7079cebb37b0d1" exitCode=0 Jul 07 17:48:26.833953 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:26.833860 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" event={"ID":"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d","Type":"ContainerDied","Data":"5e30484140cb2c26f5669209626cc2f04d91d0739a1680e5cd7079cebb37b0d1"} Jul 07 17:48:26.836391 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:26.836365 2577 generic.go:356] "Generic (PLEG): container finished" podID="51e7a8d5-d250-44cc-8187-b654dc0f4f3b" containerID="73b8cd0f95df68773e58c4876b88e7e7e55b9624c2076563474df81b42cb0985" exitCode=0 Jul 07 17:48:26.836493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:26.836432 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" event={"ID":"51e7a8d5-d250-44cc-8187-b654dc0f4f3b","Type":"ContainerDied","Data":"73b8cd0f95df68773e58c4876b88e7e7e55b9624c2076563474df81b42cb0985"} Jul 07 17:48:26.976576 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:26.976547 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" Jul 07 17:48:26.998848 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:26.998816 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2rqp\" (UniqueName: \"kubernetes.io/projected/794029e6-db7c-4afd-b417-e66d0fdeb1df-kube-api-access-z2rqp\") pod \"794029e6-db7c-4afd-b417-e66d0fdeb1df\" (UID: \"794029e6-db7c-4afd-b417-e66d0fdeb1df\") " Jul 07 17:48:26.999060 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:26.998870 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/794029e6-db7c-4afd-b417-e66d0fdeb1df-util\") pod \"794029e6-db7c-4afd-b417-e66d0fdeb1df\" (UID: \"794029e6-db7c-4afd-b417-e66d0fdeb1df\") " Jul 07 17:48:26.999060 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:26.998924 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/794029e6-db7c-4afd-b417-e66d0fdeb1df-bundle\") pod \"794029e6-db7c-4afd-b417-e66d0fdeb1df\" (UID: \"794029e6-db7c-4afd-b417-e66d0fdeb1df\") " Jul 07 17:48:26.999790 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:26.999757 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/794029e6-db7c-4afd-b417-e66d0fdeb1df-bundle" (OuterVolumeSpecName: "bundle") pod "794029e6-db7c-4afd-b417-e66d0fdeb1df" (UID: "794029e6-db7c-4afd-b417-e66d0fdeb1df"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:48:27.002283 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:27.002247 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/794029e6-db7c-4afd-b417-e66d0fdeb1df-kube-api-access-z2rqp" (OuterVolumeSpecName: "kube-api-access-z2rqp") pod "794029e6-db7c-4afd-b417-e66d0fdeb1df" (UID: "794029e6-db7c-4afd-b417-e66d0fdeb1df"). InnerVolumeSpecName "kube-api-access-z2rqp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:48:27.006752 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:27.006711 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/794029e6-db7c-4afd-b417-e66d0fdeb1df-util" (OuterVolumeSpecName: "util") pod "794029e6-db7c-4afd-b417-e66d0fdeb1df" (UID: "794029e6-db7c-4afd-b417-e66d0fdeb1df"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:48:27.100298 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:27.100202 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/794029e6-db7c-4afd-b417-e66d0fdeb1df-util\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:48:27.100298 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:27.100237 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/794029e6-db7c-4afd-b417-e66d0fdeb1df-bundle\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:48:27.100298 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:27.100247 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2rqp\" (UniqueName: \"kubernetes.io/projected/794029e6-db7c-4afd-b417-e66d0fdeb1df-kube-api-access-z2rqp\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:48:27.843894 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:27.843855 2577 generic.go:356] "Generic (PLEG): container finished" podID="b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d" containerID="b084fe4e390b41c53f54d352eba41ab10727c989914cb4605cb41eaee872a16c" exitCode=0 Jul 07 17:48:27.844315 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:27.843932 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" event={"ID":"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d","Type":"ContainerDied","Data":"b084fe4e390b41c53f54d352eba41ab10727c989914cb4605cb41eaee872a16c"} Jul 07 17:48:27.846056 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:27.846016 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" event={"ID":"794029e6-db7c-4afd-b417-e66d0fdeb1df","Type":"ContainerDied","Data":"983741134db47ab1acfca35208c48a325eea0ae19f22cee43206c3ff06373e58"} Jul 07 17:48:27.846163 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:27.846082 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="983741134db47ab1acfca35208c48a325eea0ae19f22cee43206c3ff06373e58" Jul 07 17:48:27.846838 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:27.846347 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq" Jul 07 17:48:27.988832 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:27.988802 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" Jul 07 17:48:28.019462 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.019427 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" Jul 07 17:48:28.108475 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.108359 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c00d28e6-3bfc-4192-b25f-52868d7a2194-util\") pod \"c00d28e6-3bfc-4192-b25f-52868d7a2194\" (UID: \"c00d28e6-3bfc-4192-b25f-52868d7a2194\") " Jul 07 17:48:28.108475 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.108421 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-bundle\") pod \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\" (UID: \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\") " Jul 07 17:48:28.108475 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.108463 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c00d28e6-3bfc-4192-b25f-52868d7a2194-bundle\") pod \"c00d28e6-3bfc-4192-b25f-52868d7a2194\" (UID: \"c00d28e6-3bfc-4192-b25f-52868d7a2194\") " Jul 07 17:48:28.108726 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.108500 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62vkk\" (UniqueName: \"kubernetes.io/projected/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-kube-api-access-62vkk\") pod \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\" (UID: \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\") " Jul 07 17:48:28.108726 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.108545 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-util\") pod \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\" (UID: \"51e7a8d5-d250-44cc-8187-b654dc0f4f3b\") " Jul 07 17:48:28.108726 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.108580 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrs86\" (UniqueName: \"kubernetes.io/projected/c00d28e6-3bfc-4192-b25f-52868d7a2194-kube-api-access-nrs86\") pod \"c00d28e6-3bfc-4192-b25f-52868d7a2194\" (UID: \"c00d28e6-3bfc-4192-b25f-52868d7a2194\") " Jul 07 17:48:28.109026 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.108995 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-bundle" (OuterVolumeSpecName: "bundle") pod "51e7a8d5-d250-44cc-8187-b654dc0f4f3b" (UID: "51e7a8d5-d250-44cc-8187-b654dc0f4f3b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:48:28.109166 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.108992 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c00d28e6-3bfc-4192-b25f-52868d7a2194-bundle" (OuterVolumeSpecName: "bundle") pod "c00d28e6-3bfc-4192-b25f-52868d7a2194" (UID: "c00d28e6-3bfc-4192-b25f-52868d7a2194"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:48:28.110969 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.110937 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c00d28e6-3bfc-4192-b25f-52868d7a2194-kube-api-access-nrs86" (OuterVolumeSpecName: "kube-api-access-nrs86") pod "c00d28e6-3bfc-4192-b25f-52868d7a2194" (UID: "c00d28e6-3bfc-4192-b25f-52868d7a2194"). InnerVolumeSpecName "kube-api-access-nrs86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:48:28.111103 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.111021 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-kube-api-access-62vkk" (OuterVolumeSpecName: "kube-api-access-62vkk") pod "51e7a8d5-d250-44cc-8187-b654dc0f4f3b" (UID: "51e7a8d5-d250-44cc-8187-b654dc0f4f3b"). InnerVolumeSpecName "kube-api-access-62vkk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:48:28.117322 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.117283 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c00d28e6-3bfc-4192-b25f-52868d7a2194-util" (OuterVolumeSpecName: "util") pod "c00d28e6-3bfc-4192-b25f-52868d7a2194" (UID: "c00d28e6-3bfc-4192-b25f-52868d7a2194"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:48:28.117661 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.117640 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-util" (OuterVolumeSpecName: "util") pod "51e7a8d5-d250-44cc-8187-b654dc0f4f3b" (UID: "51e7a8d5-d250-44cc-8187-b654dc0f4f3b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:48:28.209144 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.209109 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c00d28e6-3bfc-4192-b25f-52868d7a2194-bundle\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:48:28.209144 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.209142 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62vkk\" (UniqueName: \"kubernetes.io/projected/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-kube-api-access-62vkk\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:48:28.209358 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.209155 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-util\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:48:28.209358 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.209165 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrs86\" (UniqueName: \"kubernetes.io/projected/c00d28e6-3bfc-4192-b25f-52868d7a2194-kube-api-access-nrs86\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:48:28.209358 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.209177 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c00d28e6-3bfc-4192-b25f-52868d7a2194-util\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:48:28.209358 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.209189 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51e7a8d5-d250-44cc-8187-b654dc0f4f3b-bundle\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:48:28.853127 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.853009 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" Jul 07 17:48:28.853127 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.853056 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs" event={"ID":"51e7a8d5-d250-44cc-8187-b654dc0f4f3b","Type":"ContainerDied","Data":"d175dc884e4f68cc446d4e9447487ac8148df1967fdcee45c454e372c555f82a"} Jul 07 17:48:28.853127 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.853108 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d175dc884e4f68cc446d4e9447487ac8148df1967fdcee45c454e372c555f82a" Jul 07 17:48:28.855483 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.855370 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" event={"ID":"c00d28e6-3bfc-4192-b25f-52868d7a2194","Type":"ContainerDied","Data":"ad45f10249af12db9b3a624058555f9ab2c200858b47da8cb24b36545baf4088"} Jul 07 17:48:28.855483 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.855428 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad45f10249af12db9b3a624058555f9ab2c200858b47da8cb24b36545baf4088" Jul 07 17:48:28.855675 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.855648 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76" Jul 07 17:48:28.995130 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:28.995102 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" Jul 07 17:48:29.014328 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:29.014295 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-util\") pod \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\" (UID: \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\") " Jul 07 17:48:29.014532 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:29.014347 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-bundle\") pod \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\" (UID: \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\") " Jul 07 17:48:29.014532 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:29.014411 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rl4c\" (UniqueName: \"kubernetes.io/projected/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-kube-api-access-4rl4c\") pod \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\" (UID: \"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d\") " Jul 07 17:48:29.015094 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:29.015034 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-bundle" (OuterVolumeSpecName: "bundle") pod "b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d" (UID: "b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:48:29.016897 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:29.016869 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-kube-api-access-4rl4c" (OuterVolumeSpecName: "kube-api-access-4rl4c") pod "b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d" (UID: "b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d"). InnerVolumeSpecName "kube-api-access-4rl4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:48:29.020514 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:29.020479 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-util" (OuterVolumeSpecName: "util") pod "b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d" (UID: "b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 17:48:29.115104 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:29.114971 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rl4c\" (UniqueName: \"kubernetes.io/projected/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-kube-api-access-4rl4c\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:48:29.115104 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:29.115006 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-util\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:48:29.115104 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:29.115015 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d-bundle\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:48:29.863023 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:29.862997 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" Jul 07 17:48:29.863023 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:29.863014 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76" event={"ID":"b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d","Type":"ContainerDied","Data":"13e1eae5746ccc2cb5ed19c4acac588c501fffbdc7d3b6fb1f98c279ce941eb5"} Jul 07 17:48:29.863503 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:29.863081 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="13e1eae5746ccc2cb5ed19c4acac588c501fffbdc7d3b6fb1f98c279ce941eb5" Jul 07 17:48:38.435317 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435253 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-566b8fccc-cm2s9"] Jul 07 17:48:38.435826 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435703 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d" containerName="pull" Jul 07 17:48:38.435826 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435721 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d" containerName="pull" Jul 07 17:48:38.435826 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435737 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c00d28e6-3bfc-4192-b25f-52868d7a2194" containerName="extract" Jul 07 17:48:38.435826 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435745 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c00d28e6-3bfc-4192-b25f-52868d7a2194" containerName="extract" Jul 07 17:48:38.435826 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435754 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51e7a8d5-d250-44cc-8187-b654dc0f4f3b" containerName="extract" Jul 07 17:48:38.435826 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435762 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="51e7a8d5-d250-44cc-8187-b654dc0f4f3b" containerName="extract" Jul 07 17:48:38.435826 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435771 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d" containerName="extract" Jul 07 17:48:38.435826 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435779 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d" containerName="extract" Jul 07 17:48:38.435826 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435793 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51e7a8d5-d250-44cc-8187-b654dc0f4f3b" containerName="util" Jul 07 17:48:38.435826 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435800 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="51e7a8d5-d250-44cc-8187-b654dc0f4f3b" containerName="util" Jul 07 17:48:38.435826 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435810 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51e7a8d5-d250-44cc-8187-b654dc0f4f3b" containerName="pull" Jul 07 17:48:38.435826 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435817 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="51e7a8d5-d250-44cc-8187-b654dc0f4f3b" containerName="pull" Jul 07 17:48:38.435826 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435827 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c00d28e6-3bfc-4192-b25f-52868d7a2194" containerName="pull" Jul 07 17:48:38.435826 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435835 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c00d28e6-3bfc-4192-b25f-52868d7a2194" containerName="pull" Jul 07 17:48:38.436493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435847 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="794029e6-db7c-4afd-b417-e66d0fdeb1df" containerName="extract" Jul 07 17:48:38.436493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435854 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="794029e6-db7c-4afd-b417-e66d0fdeb1df" containerName="extract" Jul 07 17:48:38.436493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435871 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c00d28e6-3bfc-4192-b25f-52868d7a2194" containerName="util" Jul 07 17:48:38.436493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435879 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c00d28e6-3bfc-4192-b25f-52868d7a2194" containerName="util" Jul 07 17:48:38.436493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435886 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="794029e6-db7c-4afd-b417-e66d0fdeb1df" containerName="pull" Jul 07 17:48:38.436493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435894 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="794029e6-db7c-4afd-b417-e66d0fdeb1df" containerName="pull" Jul 07 17:48:38.436493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435912 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="794029e6-db7c-4afd-b417-e66d0fdeb1df" containerName="util" Jul 07 17:48:38.436493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435920 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="794029e6-db7c-4afd-b417-e66d0fdeb1df" containerName="util" Jul 07 17:48:38.436493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435934 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d" containerName="util" Jul 07 17:48:38.436493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.435942 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d" containerName="util" Jul 07 17:48:38.436493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.436020 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d" containerName="extract" Jul 07 17:48:38.436493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.436034 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="51e7a8d5-d250-44cc-8187-b654dc0f4f3b" containerName="extract" Jul 07 17:48:38.436493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.436068 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c00d28e6-3bfc-4192-b25f-52868d7a2194" containerName="extract" Jul 07 17:48:38.436493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.436076 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="794029e6-db7c-4afd-b417-e66d0fdeb1df" containerName="extract" Jul 07 17:48:38.439667 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.439642 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.454539 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.454506 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-566b8fccc-cm2s9"] Jul 07 17:48:38.486608 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.486556 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c43a60e4-f75c-41d2-86a1-893e37287666-console-oauth-config\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.486799 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.486662 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8gsl\" (UniqueName: \"kubernetes.io/projected/c43a60e4-f75c-41d2-86a1-893e37287666-kube-api-access-q8gsl\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.486799 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.486694 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c43a60e4-f75c-41d2-86a1-893e37287666-service-ca\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.486799 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.486729 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c43a60e4-f75c-41d2-86a1-893e37287666-console-serving-cert\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.486966 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.486819 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c43a60e4-f75c-41d2-86a1-893e37287666-console-config\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.486966 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.486856 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c43a60e4-f75c-41d2-86a1-893e37287666-oauth-serving-cert\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.486966 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.486888 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c43a60e4-f75c-41d2-86a1-893e37287666-trusted-ca-bundle\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.587554 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.587515 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c43a60e4-f75c-41d2-86a1-893e37287666-console-serving-cert\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.587758 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.587574 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c43a60e4-f75c-41d2-86a1-893e37287666-console-config\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.587758 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.587624 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c43a60e4-f75c-41d2-86a1-893e37287666-oauth-serving-cert\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.587758 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.587652 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c43a60e4-f75c-41d2-86a1-893e37287666-trusted-ca-bundle\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.587758 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.587696 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c43a60e4-f75c-41d2-86a1-893e37287666-console-oauth-config\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.587758 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.587736 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8gsl\" (UniqueName: \"kubernetes.io/projected/c43a60e4-f75c-41d2-86a1-893e37287666-kube-api-access-q8gsl\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.588090 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.587765 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c43a60e4-f75c-41d2-86a1-893e37287666-service-ca\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.588642 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.588613 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c43a60e4-f75c-41d2-86a1-893e37287666-service-ca\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.588762 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.588735 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c43a60e4-f75c-41d2-86a1-893e37287666-console-config\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.588798 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.588757 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c43a60e4-f75c-41d2-86a1-893e37287666-trusted-ca-bundle\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.588881 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.588854 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c43a60e4-f75c-41d2-86a1-893e37287666-oauth-serving-cert\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.590347 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.590321 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c43a60e4-f75c-41d2-86a1-893e37287666-console-oauth-config\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.590488 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.590468 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c43a60e4-f75c-41d2-86a1-893e37287666-console-serving-cert\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.598782 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.598752 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8gsl\" (UniqueName: \"kubernetes.io/projected/c43a60e4-f75c-41d2-86a1-893e37287666-kube-api-access-q8gsl\") pod \"console-566b8fccc-cm2s9\" (UID: \"c43a60e4-f75c-41d2-86a1-893e37287666\") " pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.750471 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.750383 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:38.904222 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:38.904174 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-566b8fccc-cm2s9"] Jul 07 17:48:38.906653 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:48:38.906624 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc43a60e4_f75c_41d2_86a1_893e37287666.slice/crio-286d3d4680ae2de8874a5fab779ff8371a58f5d475e1a40029b8a838577d8016 WatchSource:0}: Error finding container 286d3d4680ae2de8874a5fab779ff8371a58f5d475e1a40029b8a838577d8016: Status 404 returned error can't find the container with id 286d3d4680ae2de8874a5fab779ff8371a58f5d475e1a40029b8a838577d8016 Jul 07 17:48:39.913579 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:39.913543 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-566b8fccc-cm2s9" event={"ID":"c43a60e4-f75c-41d2-86a1-893e37287666","Type":"ContainerStarted","Data":"0b8db6c20e9415b7dfab1edfd6ef8fac9dab5482a2c404cb3c673462a58b7f5c"} Jul 07 17:48:39.913998 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:39.913593 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-566b8fccc-cm2s9" event={"ID":"c43a60e4-f75c-41d2-86a1-893e37287666","Type":"ContainerStarted","Data":"286d3d4680ae2de8874a5fab779ff8371a58f5d475e1a40029b8a838577d8016"} Jul 07 17:48:39.949349 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:39.949293 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-566b8fccc-cm2s9" podStartSLOduration=1.949274003 podStartE2EDuration="1.949274003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:48:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:48:39.94655268 +0000 UTC m=+523.985879583" watchObservedRunningTime="2026-07-07 17:48:39.949274003 +0000 UTC m=+523.988600905" Jul 07 17:48:48.751547 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:48.751439 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:48.751547 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:48.751515 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:48.756813 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:48.756781 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:48.963248 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:48.963216 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-566b8fccc-cm2s9" Jul 07 17:48:49.102983 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:49.102871 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9df66c4db-8zcd2"] Jul 07 17:48:58.784981 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.784937 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-722dn"] Jul 07 17:48:58.789551 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.789523 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-722dn" Jul 07 17:48:58.792785 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.792760 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 07 17:48:58.792962 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.792909 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 07 17:48:58.793028 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.792968 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 17:48:58.793812 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.793787 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 17:48:58.793990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.793971 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ptkw2\"" Jul 07 17:48:58.801012 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.800983 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-722dn"] Jul 07 17:48:58.852224 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.852166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff0b5bb9-13b4-4f08-80a0-02aeb4289c99-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-722dn\" (UID: \"ff0b5bb9-13b4-4f08-80a0-02aeb4289c99\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-722dn" Jul 07 17:48:58.852416 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.852240 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ff0b5bb9-13b4-4f08-80a0-02aeb4289c99-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-722dn\" (UID: \"ff0b5bb9-13b4-4f08-80a0-02aeb4289c99\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-722dn" Jul 07 17:48:58.852416 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.852268 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf45x\" (UniqueName: \"kubernetes.io/projected/ff0b5bb9-13b4-4f08-80a0-02aeb4289c99-kube-api-access-hf45x\") pod \"kuadrant-console-plugin-66b5774985-722dn\" (UID: \"ff0b5bb9-13b4-4f08-80a0-02aeb4289c99\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-722dn" Jul 07 17:48:58.953375 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.953336 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff0b5bb9-13b4-4f08-80a0-02aeb4289c99-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-722dn\" (UID: \"ff0b5bb9-13b4-4f08-80a0-02aeb4289c99\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-722dn" Jul 07 17:48:58.953632 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.953404 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ff0b5bb9-13b4-4f08-80a0-02aeb4289c99-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-722dn\" (UID: \"ff0b5bb9-13b4-4f08-80a0-02aeb4289c99\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-722dn" Jul 07 17:48:58.953632 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.953445 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf45x\" (UniqueName: \"kubernetes.io/projected/ff0b5bb9-13b4-4f08-80a0-02aeb4289c99-kube-api-access-hf45x\") pod \"kuadrant-console-plugin-66b5774985-722dn\" (UID: \"ff0b5bb9-13b4-4f08-80a0-02aeb4289c99\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-722dn" Jul 07 17:48:58.954300 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.954273 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ff0b5bb9-13b4-4f08-80a0-02aeb4289c99-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-722dn\" (UID: \"ff0b5bb9-13b4-4f08-80a0-02aeb4289c99\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-722dn" Jul 07 17:48:58.956207 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.956179 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff0b5bb9-13b4-4f08-80a0-02aeb4289c99-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-722dn\" (UID: \"ff0b5bb9-13b4-4f08-80a0-02aeb4289c99\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-722dn" Jul 07 17:48:58.965700 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:58.965670 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf45x\" (UniqueName: \"kubernetes.io/projected/ff0b5bb9-13b4-4f08-80a0-02aeb4289c99-kube-api-access-hf45x\") pod \"kuadrant-console-plugin-66b5774985-722dn\" (UID: \"ff0b5bb9-13b4-4f08-80a0-02aeb4289c99\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-722dn" Jul 07 17:48:59.100103 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:59.099980 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-722dn" Jul 07 17:48:59.255173 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:48:59.255144 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-722dn"] Jul 07 17:48:59.257170 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:48:59.257145 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff0b5bb9_13b4_4f08_80a0_02aeb4289c99.slice/crio-52dc8d8c7506bb094cd581edec25ecdcf26f72494820206d690b0cd77c2bc983 WatchSource:0}: Error finding container 52dc8d8c7506bb094cd581edec25ecdcf26f72494820206d690b0cd77c2bc983: Status 404 returned error can't find the container with id 52dc8d8c7506bb094cd581edec25ecdcf26f72494820206d690b0cd77c2bc983 Jul 07 17:49:00.016106 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:00.016064 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-722dn" event={"ID":"ff0b5bb9-13b4-4f08-80a0-02aeb4289c99","Type":"ContainerStarted","Data":"52dc8d8c7506bb094cd581edec25ecdcf26f72494820206d690b0cd77c2bc983"} Jul 07 17:49:14.130592 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:14.130540 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-9df66c4db-8zcd2" podUID="d15f2074-44cb-416f-b396-36faa59058d4" containerName="console" containerID="cri-o://df21aac42e8381c7f7f30341b219e0b9476bb6c7de00c7f865f4e6edb73aeee6" gracePeriod=15 Jul 07 17:49:15.766052 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:15.765999 2577 patch_prober.go:28] interesting pod/console-9df66c4db-8zcd2 container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.24:8443/health\": dial tcp 10.132.0.24:8443: connect: connection refused" start-of-body= Jul 07 17:49:15.766513 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:15.766124 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-9df66c4db-8zcd2" podUID="d15f2074-44cb-416f-b396-36faa59058d4" containerName="console" probeResult="failure" output="Get \"https://10.132.0.24:8443/health\": dial tcp 10.132.0.24:8443: connect: connection refused" Jul 07 17:49:22.444714 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.444677 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:49:22.578278 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.578179 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-trusted-ca-bundle\") pod \"d15f2074-44cb-416f-b396-36faa59058d4\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " Jul 07 17:49:22.578278 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.578247 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d15f2074-44cb-416f-b396-36faa59058d4-console-oauth-config\") pod \"d15f2074-44cb-416f-b396-36faa59058d4\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " Jul 07 17:49:22.578278 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.578274 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-service-ca\") pod \"d15f2074-44cb-416f-b396-36faa59058d4\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " Jul 07 17:49:22.578561 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.578372 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9k8dd\" (UniqueName: \"kubernetes.io/projected/d15f2074-44cb-416f-b396-36faa59058d4-kube-api-access-9k8dd\") pod \"d15f2074-44cb-416f-b396-36faa59058d4\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " Jul 07 17:49:22.578561 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.578412 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-console-config\") pod \"d15f2074-44cb-416f-b396-36faa59058d4\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " Jul 07 17:49:22.578561 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.578461 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d15f2074-44cb-416f-b396-36faa59058d4-console-serving-cert\") pod \"d15f2074-44cb-416f-b396-36faa59058d4\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " Jul 07 17:49:22.578561 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.578501 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-oauth-serving-cert\") pod \"d15f2074-44cb-416f-b396-36faa59058d4\" (UID: \"d15f2074-44cb-416f-b396-36faa59058d4\") " Jul 07 17:49:22.578838 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.578675 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-service-ca" (OuterVolumeSpecName: "service-ca") pod "d15f2074-44cb-416f-b396-36faa59058d4" (UID: "d15f2074-44cb-416f-b396-36faa59058d4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:49:22.578838 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.578797 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d15f2074-44cb-416f-b396-36faa59058d4" (UID: "d15f2074-44cb-416f-b396-36faa59058d4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:49:22.579013 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.578833 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-console-config" (OuterVolumeSpecName: "console-config") pod "d15f2074-44cb-416f-b396-36faa59058d4" (UID: "d15f2074-44cb-416f-b396-36faa59058d4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:49:22.579111 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.579078 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d15f2074-44cb-416f-b396-36faa59058d4" (UID: "d15f2074-44cb-416f-b396-36faa59058d4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:49:22.580722 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.580686 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d15f2074-44cb-416f-b396-36faa59058d4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d15f2074-44cb-416f-b396-36faa59058d4" (UID: "d15f2074-44cb-416f-b396-36faa59058d4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:49:22.580849 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.580734 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d15f2074-44cb-416f-b396-36faa59058d4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d15f2074-44cb-416f-b396-36faa59058d4" (UID: "d15f2074-44cb-416f-b396-36faa59058d4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:49:22.580849 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.580748 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d15f2074-44cb-416f-b396-36faa59058d4-kube-api-access-9k8dd" (OuterVolumeSpecName: "kube-api-access-9k8dd") pod "d15f2074-44cb-416f-b396-36faa59058d4" (UID: "d15f2074-44cb-416f-b396-36faa59058d4"). InnerVolumeSpecName "kube-api-access-9k8dd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:49:22.680228 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.680193 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-trusted-ca-bundle\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:49:22.680228 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.680224 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d15f2074-44cb-416f-b396-36faa59058d4-console-oauth-config\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:49:22.680228 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.680234 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:49:22.680470 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.680245 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9k8dd\" (UniqueName: \"kubernetes.io/projected/d15f2074-44cb-416f-b396-36faa59058d4-kube-api-access-9k8dd\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:49:22.680470 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.680259 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-console-config\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:49:22.680470 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.680270 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d15f2074-44cb-416f-b396-36faa59058d4-console-serving-cert\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:49:22.680470 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:22.680278 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d15f2074-44cb-416f-b396-36faa59058d4-oauth-serving-cert\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:49:23.156349 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:23.156300 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-722dn" event={"ID":"ff0b5bb9-13b4-4f08-80a0-02aeb4289c99","Type":"ContainerStarted","Data":"6b521f5faf8cb05abbba30c420f49c99a16efc03b18e220ffbd454a464c10136"} Jul 07 17:49:23.157730 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:23.157697 2577 generic.go:356] "Generic (PLEG): container finished" podID="d15f2074-44cb-416f-b396-36faa59058d4" containerID="df21aac42e8381c7f7f30341b219e0b9476bb6c7de00c7f865f4e6edb73aeee6" exitCode=0 Jul 07 17:49:23.157867 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:23.157745 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9df66c4db-8zcd2" Jul 07 17:49:23.157867 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:23.157756 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9df66c4db-8zcd2" event={"ID":"d15f2074-44cb-416f-b396-36faa59058d4","Type":"ContainerDied","Data":"df21aac42e8381c7f7f30341b219e0b9476bb6c7de00c7f865f4e6edb73aeee6"} Jul 07 17:49:23.157867 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:23.157813 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9df66c4db-8zcd2" event={"ID":"d15f2074-44cb-416f-b396-36faa59058d4","Type":"ContainerDied","Data":"11f65f0280e950649954228413fc4be565518e962252f38e661d3dfea17271be"} Jul 07 17:49:23.157867 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:23.157840 2577 scope.go:117] "RemoveContainer" containerID="df21aac42e8381c7f7f30341b219e0b9476bb6c7de00c7f865f4e6edb73aeee6" Jul 07 17:49:23.167813 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:23.167791 2577 scope.go:117] "RemoveContainer" containerID="df21aac42e8381c7f7f30341b219e0b9476bb6c7de00c7f865f4e6edb73aeee6" Jul 07 17:49:23.168187 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:49:23.168159 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df21aac42e8381c7f7f30341b219e0b9476bb6c7de00c7f865f4e6edb73aeee6\": container with ID starting with df21aac42e8381c7f7f30341b219e0b9476bb6c7de00c7f865f4e6edb73aeee6 not found: ID does not exist" containerID="df21aac42e8381c7f7f30341b219e0b9476bb6c7de00c7f865f4e6edb73aeee6" Jul 07 17:49:23.168275 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:23.168196 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df21aac42e8381c7f7f30341b219e0b9476bb6c7de00c7f865f4e6edb73aeee6"} err="failed to get container status \"df21aac42e8381c7f7f30341b219e0b9476bb6c7de00c7f865f4e6edb73aeee6\": rpc error: code = NotFound desc = could not find container \"df21aac42e8381c7f7f30341b219e0b9476bb6c7de00c7f865f4e6edb73aeee6\": container with ID starting with df21aac42e8381c7f7f30341b219e0b9476bb6c7de00c7f865f4e6edb73aeee6 not found: ID does not exist" Jul 07 17:49:23.175210 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:23.175159 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-722dn" podStartSLOduration=1.742609441 podStartE2EDuration="25.175142422s" totalImagesPullingTime="23.432532981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:49:23.174657633 +0000 UTC m=+567.213984539" watchObservedRunningTime="2026-07-07 17:49:23.175142422 +0000 UTC m=+567.214469324" Jul 07 17:49:23.194453 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:23.194407 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9df66c4db-8zcd2"] Jul 07 17:49:23.201385 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:23.201353 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9df66c4db-8zcd2"] Jul 07 17:49:24.522238 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:24.522200 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d15f2074-44cb-416f-b396-36faa59058d4" path="/var/lib/kubelet/pods/d15f2074-44cb-416f-b396-36faa59058d4/volumes" Jul 07 17:49:56.444909 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:56.444879 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 17:49:56.451383 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:56.451354 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 17:49:57.304371 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.304332 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rxgs4"] Jul 07 17:49:57.304662 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.304648 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d15f2074-44cb-416f-b396-36faa59058d4" containerName="console" Jul 07 17:49:57.304706 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.304663 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d15f2074-44cb-416f-b396-36faa59058d4" containerName="console" Jul 07 17:49:57.304740 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.304728 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d15f2074-44cb-416f-b396-36faa59058d4" containerName="console" Jul 07 17:49:57.362649 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.362607 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rxgs4"] Jul 07 17:49:57.362836 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.362735 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" Jul 07 17:49:57.365490 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.365465 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 17:49:57.401660 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.401625 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rxgs4"] Jul 07 17:49:57.485966 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.485932 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6b520ab6-40c1-44ba-99f6-fe7d0220731f-config-file\") pod \"limitador-limitador-64c94fd64f-rxgs4\" (UID: \"6b520ab6-40c1-44ba-99f6-fe7d0220731f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" Jul 07 17:49:57.486377 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.485996 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thtj6\" (UniqueName: \"kubernetes.io/projected/6b520ab6-40c1-44ba-99f6-fe7d0220731f-kube-api-access-thtj6\") pod \"limitador-limitador-64c94fd64f-rxgs4\" (UID: \"6b520ab6-40c1-44ba-99f6-fe7d0220731f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" Jul 07 17:49:57.587198 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.587099 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6b520ab6-40c1-44ba-99f6-fe7d0220731f-config-file\") pod \"limitador-limitador-64c94fd64f-rxgs4\" (UID: \"6b520ab6-40c1-44ba-99f6-fe7d0220731f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" Jul 07 17:49:57.587198 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.587158 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thtj6\" (UniqueName: \"kubernetes.io/projected/6b520ab6-40c1-44ba-99f6-fe7d0220731f-kube-api-access-thtj6\") pod \"limitador-limitador-64c94fd64f-rxgs4\" (UID: \"6b520ab6-40c1-44ba-99f6-fe7d0220731f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" Jul 07 17:49:57.587820 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.587792 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6b520ab6-40c1-44ba-99f6-fe7d0220731f-config-file\") pod \"limitador-limitador-64c94fd64f-rxgs4\" (UID: \"6b520ab6-40c1-44ba-99f6-fe7d0220731f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" Jul 07 17:49:57.596436 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.596398 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thtj6\" (UniqueName: \"kubernetes.io/projected/6b520ab6-40c1-44ba-99f6-fe7d0220731f-kube-api-access-thtj6\") pod \"limitador-limitador-64c94fd64f-rxgs4\" (UID: \"6b520ab6-40c1-44ba-99f6-fe7d0220731f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" Jul 07 17:49:57.674418 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.674374 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" Jul 07 17:49:57.826776 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:57.826729 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rxgs4"] Jul 07 17:49:57.829138 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:49:57.829099 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b520ab6_40c1_44ba_99f6_fe7d0220731f.slice/crio-ab299fdbf2dae31d709d23a6aa7242ec8551802add441056f570ddc6e1724931 WatchSource:0}: Error finding container ab299fdbf2dae31d709d23a6aa7242ec8551802add441056f570ddc6e1724931: Status 404 returned error can't find the container with id ab299fdbf2dae31d709d23a6aa7242ec8551802add441056f570ddc6e1724931 Jul 07 17:49:58.127648 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.127610 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-kvkzv"] Jul 07 17:49:58.132654 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.132629 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kvkzv" Jul 07 17:49:58.135011 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.134990 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rzbhw\"" Jul 07 17:49:58.144138 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.144099 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-kvkzv"] Jul 07 17:49:58.193889 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.193844 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cch72\" (UniqueName: \"kubernetes.io/projected/7160d4bb-3970-42f8-89b5-75c8c58e624b-kube-api-access-cch72\") pod \"authorino-58cd955675-kvkzv\" (UID: \"7160d4bb-3970-42f8-89b5-75c8c58e624b\") " pod="kuadrant-system/authorino-58cd955675-kvkzv" Jul 07 17:49:58.286391 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.286350 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-t69jl"] Jul 07 17:49:58.289845 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.289822 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t69jl" Jul 07 17:49:58.294414 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.294392 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cch72\" (UniqueName: \"kubernetes.io/projected/7160d4bb-3970-42f8-89b5-75c8c58e624b-kube-api-access-cch72\") pod \"authorino-58cd955675-kvkzv\" (UID: \"7160d4bb-3970-42f8-89b5-75c8c58e624b\") " pod="kuadrant-system/authorino-58cd955675-kvkzv" Jul 07 17:49:58.301645 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.301611 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t69jl"] Jul 07 17:49:58.305155 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.305121 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cch72\" (UniqueName: \"kubernetes.io/projected/7160d4bb-3970-42f8-89b5-75c8c58e624b-kube-api-access-cch72\") pod \"authorino-58cd955675-kvkzv\" (UID: \"7160d4bb-3970-42f8-89b5-75c8c58e624b\") " pod="kuadrant-system/authorino-58cd955675-kvkzv" Jul 07 17:49:58.344708 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.344665 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" event={"ID":"6b520ab6-40c1-44ba-99f6-fe7d0220731f","Type":"ContainerStarted","Data":"ab299fdbf2dae31d709d23a6aa7242ec8551802add441056f570ddc6e1724931"} Jul 07 17:49:58.395475 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.395375 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9jnx\" (UniqueName: \"kubernetes.io/projected/e25b0658-7739-4952-8f2d-6c2a3b893de7-kube-api-access-t9jnx\") pod \"authorino-75d556bd77-t69jl\" (UID: \"e25b0658-7739-4952-8f2d-6c2a3b893de7\") " pod="kuadrant-system/authorino-75d556bd77-t69jl" Jul 07 17:49:58.445468 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.445433 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kvkzv" Jul 07 17:49:58.496867 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.496820 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9jnx\" (UniqueName: \"kubernetes.io/projected/e25b0658-7739-4952-8f2d-6c2a3b893de7-kube-api-access-t9jnx\") pod \"authorino-75d556bd77-t69jl\" (UID: \"e25b0658-7739-4952-8f2d-6c2a3b893de7\") " pod="kuadrant-system/authorino-75d556bd77-t69jl" Jul 07 17:49:58.506288 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.506231 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9jnx\" (UniqueName: \"kubernetes.io/projected/e25b0658-7739-4952-8f2d-6c2a3b893de7-kube-api-access-t9jnx\") pod \"authorino-75d556bd77-t69jl\" (UID: \"e25b0658-7739-4952-8f2d-6c2a3b893de7\") " pod="kuadrant-system/authorino-75d556bd77-t69jl" Jul 07 17:49:58.600337 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.600020 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t69jl" Jul 07 17:49:58.634953 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.634560 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-kvkzv"] Jul 07 17:49:58.636827 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:49:58.636777 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7160d4bb_3970_42f8_89b5_75c8c58e624b.slice/crio-3de38d2e7089f7b1cb4886cb6b03c4082b462620afc35b8bb59697d07ef122a8 WatchSource:0}: Error finding container 3de38d2e7089f7b1cb4886cb6b03c4082b462620afc35b8bb59697d07ef122a8: Status 404 returned error can't find the container with id 3de38d2e7089f7b1cb4886cb6b03c4082b462620afc35b8bb59697d07ef122a8 Jul 07 17:49:58.827516 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:58.826875 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t69jl"] Jul 07 17:49:58.830139 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:49:58.830097 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode25b0658_7739_4952_8f2d_6c2a3b893de7.slice/crio-7bbb8e2c3d3c3736b12fed91678d8d134b6aec446c10932e7bdcf1dd41313798 WatchSource:0}: Error finding container 7bbb8e2c3d3c3736b12fed91678d8d134b6aec446c10932e7bdcf1dd41313798: Status 404 returned error can't find the container with id 7bbb8e2c3d3c3736b12fed91678d8d134b6aec446c10932e7bdcf1dd41313798 Jul 07 17:49:59.352407 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:59.352369 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kvkzv" event={"ID":"7160d4bb-3970-42f8-89b5-75c8c58e624b","Type":"ContainerStarted","Data":"3de38d2e7089f7b1cb4886cb6b03c4082b462620afc35b8bb59697d07ef122a8"} Jul 07 17:49:59.353737 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:49:59.353699 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t69jl" event={"ID":"e25b0658-7739-4952-8f2d-6c2a3b893de7","Type":"ContainerStarted","Data":"7bbb8e2c3d3c3736b12fed91678d8d134b6aec446c10932e7bdcf1dd41313798"} Jul 07 17:50:01.385099 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:01.384984 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" event={"ID":"6b520ab6-40c1-44ba-99f6-fe7d0220731f","Type":"ContainerStarted","Data":"01af2708ed85a854f1968a33140a05f390812aa495d24cd6bbbdeb476b88c9cd"} Jul 07 17:50:01.385865 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:01.385835 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" Jul 07 17:50:01.413226 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:01.413155 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" podStartSLOduration=1.312525578 podStartE2EDuration="4.413126841s" totalImagesPullingTime="3.100601263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:49:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:50:01.409520213 +0000 UTC m=+605.448847121" watchObservedRunningTime="2026-07-07 17:50:01.413126841 +0000 UTC m=+605.452453745" Jul 07 17:50:03.398958 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:03.398917 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kvkzv" event={"ID":"7160d4bb-3970-42f8-89b5-75c8c58e624b","Type":"ContainerStarted","Data":"cc7430918c43579fd3a8816f4890f7ffa32adfad391ef7821ad464ccd2832e08"} Jul 07 17:50:03.400691 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:03.400656 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t69jl" event={"ID":"e25b0658-7739-4952-8f2d-6c2a3b893de7","Type":"ContainerStarted","Data":"df14b0b08d97de97c83cb6c231a60d34c3f5eab7efc77ac1fda5c509193c8606"} Jul 07 17:50:03.415816 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:03.415750 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-kvkzv" podStartSLOduration=0.858925053 podStartE2EDuration="5.415732153s" totalImagesPullingTime="4.5568071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:50:03.41360237 +0000 UTC m=+607.452929275" watchObservedRunningTime="2026-07-07 17:50:03.415732153 +0000 UTC m=+607.455059055" Jul 07 17:50:03.429402 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:03.429349 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-t69jl" podStartSLOduration=1.078097969 podStartE2EDuration="5.429328893s" totalImagesPullingTime="4.351230924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:50:03.428259211 +0000 UTC m=+607.467586116" watchObservedRunningTime="2026-07-07 17:50:03.429328893 +0000 UTC m=+607.468655795" Jul 07 17:50:03.456019 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:03.455971 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-kvkzv"] Jul 07 17:50:05.411121 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:05.411052 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-kvkzv" podUID="7160d4bb-3970-42f8-89b5-75c8c58e624b" containerName="authorino" containerID="cri-o://cc7430918c43579fd3a8816f4890f7ffa32adfad391ef7821ad464ccd2832e08" gracePeriod=30 Jul 07 17:50:05.667852 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:05.667778 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kvkzv" Jul 07 17:50:05.766171 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:05.766132 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cch72\" (UniqueName: \"kubernetes.io/projected/7160d4bb-3970-42f8-89b5-75c8c58e624b-kube-api-access-cch72\") pod \"7160d4bb-3970-42f8-89b5-75c8c58e624b\" (UID: \"7160d4bb-3970-42f8-89b5-75c8c58e624b\") " Jul 07 17:50:05.768406 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:05.768368 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7160d4bb-3970-42f8-89b5-75c8c58e624b-kube-api-access-cch72" (OuterVolumeSpecName: "kube-api-access-cch72") pod "7160d4bb-3970-42f8-89b5-75c8c58e624b" (UID: "7160d4bb-3970-42f8-89b5-75c8c58e624b"). InnerVolumeSpecName "kube-api-access-cch72". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:50:05.867410 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:05.867371 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cch72\" (UniqueName: \"kubernetes.io/projected/7160d4bb-3970-42f8-89b5-75c8c58e624b-kube-api-access-cch72\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:50:06.418015 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:06.417977 2577 generic.go:356] "Generic (PLEG): container finished" podID="7160d4bb-3970-42f8-89b5-75c8c58e624b" containerID="cc7430918c43579fd3a8816f4890f7ffa32adfad391ef7821ad464ccd2832e08" exitCode=0 Jul 07 17:50:06.418498 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:06.418023 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kvkzv" Jul 07 17:50:06.418498 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:06.418069 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kvkzv" event={"ID":"7160d4bb-3970-42f8-89b5-75c8c58e624b","Type":"ContainerDied","Data":"cc7430918c43579fd3a8816f4890f7ffa32adfad391ef7821ad464ccd2832e08"} Jul 07 17:50:06.418498 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:06.418117 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kvkzv" event={"ID":"7160d4bb-3970-42f8-89b5-75c8c58e624b","Type":"ContainerDied","Data":"3de38d2e7089f7b1cb4886cb6b03c4082b462620afc35b8bb59697d07ef122a8"} Jul 07 17:50:06.418498 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:06.418134 2577 scope.go:117] "RemoveContainer" containerID="cc7430918c43579fd3a8816f4890f7ffa32adfad391ef7821ad464ccd2832e08" Jul 07 17:50:06.427933 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:06.427912 2577 scope.go:117] "RemoveContainer" containerID="cc7430918c43579fd3a8816f4890f7ffa32adfad391ef7821ad464ccd2832e08" Jul 07 17:50:06.428317 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:50:06.428296 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc7430918c43579fd3a8816f4890f7ffa32adfad391ef7821ad464ccd2832e08\": container with ID starting with cc7430918c43579fd3a8816f4890f7ffa32adfad391ef7821ad464ccd2832e08 not found: ID does not exist" containerID="cc7430918c43579fd3a8816f4890f7ffa32adfad391ef7821ad464ccd2832e08" Jul 07 17:50:06.428411 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:06.428325 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc7430918c43579fd3a8816f4890f7ffa32adfad391ef7821ad464ccd2832e08"} err="failed to get container status \"cc7430918c43579fd3a8816f4890f7ffa32adfad391ef7821ad464ccd2832e08\": rpc error: code = NotFound desc = could not find container \"cc7430918c43579fd3a8816f4890f7ffa32adfad391ef7821ad464ccd2832e08\": container with ID starting with cc7430918c43579fd3a8816f4890f7ffa32adfad391ef7821ad464ccd2832e08 not found: ID does not exist" Jul 07 17:50:06.439243 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:06.439205 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-kvkzv"] Jul 07 17:50:06.443215 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:06.443176 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-kvkzv"] Jul 07 17:50:06.515981 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:06.515939 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7160d4bb-3970-42f8-89b5-75c8c58e624b" path="/var/lib/kubelet/pods/7160d4bb-3970-42f8-89b5-75c8c58e624b/volumes" Jul 07 17:50:12.474711 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:12.474668 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rxgs4"] Jul 07 17:50:12.477329 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:12.474949 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" podUID="6b520ab6-40c1-44ba-99f6-fe7d0220731f" containerName="limitador" containerID="cri-o://01af2708ed85a854f1968a33140a05f390812aa495d24cd6bbbdeb476b88c9cd" gracePeriod=30 Jul 07 17:50:12.477329 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:12.475648 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" Jul 07 17:50:13.043762 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.043684 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" Jul 07 17:50:13.138272 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.138237 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6b520ab6-40c1-44ba-99f6-fe7d0220731f-config-file\") pod \"6b520ab6-40c1-44ba-99f6-fe7d0220731f\" (UID: \"6b520ab6-40c1-44ba-99f6-fe7d0220731f\") " Jul 07 17:50:13.138272 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.138280 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thtj6\" (UniqueName: \"kubernetes.io/projected/6b520ab6-40c1-44ba-99f6-fe7d0220731f-kube-api-access-thtj6\") pod \"6b520ab6-40c1-44ba-99f6-fe7d0220731f\" (UID: \"6b520ab6-40c1-44ba-99f6-fe7d0220731f\") " Jul 07 17:50:13.138635 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.138613 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b520ab6-40c1-44ba-99f6-fe7d0220731f-config-file" (OuterVolumeSpecName: "config-file") pod "6b520ab6-40c1-44ba-99f6-fe7d0220731f" (UID: "6b520ab6-40c1-44ba-99f6-fe7d0220731f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 17:50:13.140623 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.140598 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b520ab6-40c1-44ba-99f6-fe7d0220731f-kube-api-access-thtj6" (OuterVolumeSpecName: "kube-api-access-thtj6") pod "6b520ab6-40c1-44ba-99f6-fe7d0220731f" (UID: "6b520ab6-40c1-44ba-99f6-fe7d0220731f"). InnerVolumeSpecName "kube-api-access-thtj6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:50:13.239191 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.239155 2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6b520ab6-40c1-44ba-99f6-fe7d0220731f-config-file\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:50:13.239191 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.239188 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thtj6\" (UniqueName: \"kubernetes.io/projected/6b520ab6-40c1-44ba-99f6-fe7d0220731f-kube-api-access-thtj6\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:50:13.457986 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.457945 2577 generic.go:356] "Generic (PLEG): container finished" podID="6b520ab6-40c1-44ba-99f6-fe7d0220731f" containerID="01af2708ed85a854f1968a33140a05f390812aa495d24cd6bbbdeb476b88c9cd" exitCode=0 Jul 07 17:50:13.458220 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.458000 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" Jul 07 17:50:13.458220 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.458067 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" event={"ID":"6b520ab6-40c1-44ba-99f6-fe7d0220731f","Type":"ContainerDied","Data":"01af2708ed85a854f1968a33140a05f390812aa495d24cd6bbbdeb476b88c9cd"} Jul 07 17:50:13.458220 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.458127 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rxgs4" event={"ID":"6b520ab6-40c1-44ba-99f6-fe7d0220731f","Type":"ContainerDied","Data":"ab299fdbf2dae31d709d23a6aa7242ec8551802add441056f570ddc6e1724931"} Jul 07 17:50:13.458220 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.458151 2577 scope.go:117] "RemoveContainer" containerID="01af2708ed85a854f1968a33140a05f390812aa495d24cd6bbbdeb476b88c9cd" Jul 07 17:50:13.467769 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.467750 2577 scope.go:117] "RemoveContainer" containerID="01af2708ed85a854f1968a33140a05f390812aa495d24cd6bbbdeb476b88c9cd" Jul 07 17:50:13.468119 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:50:13.468101 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01af2708ed85a854f1968a33140a05f390812aa495d24cd6bbbdeb476b88c9cd\": container with ID starting with 01af2708ed85a854f1968a33140a05f390812aa495d24cd6bbbdeb476b88c9cd not found: ID does not exist" containerID="01af2708ed85a854f1968a33140a05f390812aa495d24cd6bbbdeb476b88c9cd" Jul 07 17:50:13.468187 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.468129 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01af2708ed85a854f1968a33140a05f390812aa495d24cd6bbbdeb476b88c9cd"} err="failed to get container status \"01af2708ed85a854f1968a33140a05f390812aa495d24cd6bbbdeb476b88c9cd\": rpc error: code = NotFound desc = could not find container \"01af2708ed85a854f1968a33140a05f390812aa495d24cd6bbbdeb476b88c9cd\": container with ID starting with 01af2708ed85a854f1968a33140a05f390812aa495d24cd6bbbdeb476b88c9cd not found: ID does not exist" Jul 07 17:50:13.478945 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.478906 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rxgs4"] Jul 07 17:50:13.485159 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:13.485122 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rxgs4"] Jul 07 17:50:14.516462 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:14.516428 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b520ab6-40c1-44ba-99f6-fe7d0220731f" path="/var/lib/kubelet/pods/6b520ab6-40c1-44ba-99f6-fe7d0220731f/volumes" Jul 07 17:50:28.359668 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.359583 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9rbtg"] Jul 07 17:50:28.360140 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.359980 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7160d4bb-3970-42f8-89b5-75c8c58e624b" containerName="authorino" Jul 07 17:50:28.360140 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.359994 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7160d4bb-3970-42f8-89b5-75c8c58e624b" containerName="authorino" Jul 07 17:50:28.360140 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.360004 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b520ab6-40c1-44ba-99f6-fe7d0220731f" containerName="limitador" Jul 07 17:50:28.360140 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.360009 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b520ab6-40c1-44ba-99f6-fe7d0220731f" containerName="limitador" Jul 07 17:50:28.360140 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.360088 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7160d4bb-3970-42f8-89b5-75c8c58e624b" containerName="authorino" Jul 07 17:50:28.360140 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.360096 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b520ab6-40c1-44ba-99f6-fe7d0220731f" containerName="limitador" Jul 07 17:50:28.362989 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.362967 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-9rbtg" Jul 07 17:50:28.371549 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.371518 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9rbtg"] Jul 07 17:50:28.463269 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.463231 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mqpm\" (UniqueName: \"kubernetes.io/projected/d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9-kube-api-access-2mqpm\") pod \"authorino-5d4b67c446-9rbtg\" (UID: \"d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9\") " pod="kuadrant-system/authorino-5d4b67c446-9rbtg" Jul 07 17:50:28.544908 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.544867 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9rbtg"] Jul 07 17:50:28.545214 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:50:28.545189 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-2mqpm], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-9rbtg" podUID="d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9" Jul 07 17:50:28.564339 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.564260 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mqpm\" (UniqueName: \"kubernetes.io/projected/d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9-kube-api-access-2mqpm\") pod \"authorino-5d4b67c446-9rbtg\" (UID: \"d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9\") " pod="kuadrant-system/authorino-5d4b67c446-9rbtg" Jul 07 17:50:28.578504 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.578471 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mqpm\" (UniqueName: \"kubernetes.io/projected/d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9-kube-api-access-2mqpm\") pod \"authorino-5d4b67c446-9rbtg\" (UID: \"d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9\") " pod="kuadrant-system/authorino-5d4b67c446-9rbtg" Jul 07 17:50:28.591737 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.591693 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-7tdb9"] Jul 07 17:50:28.595249 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.595221 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-7tdb9" Jul 07 17:50:28.599311 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.599278 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 07 17:50:28.611231 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.611136 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-7tdb9"] Jul 07 17:50:28.665564 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.665517 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58ec3eb9-233e-470e-82bc-7016926a5acc-tls-cert\") pod \"authorino-76877994-7tdb9\" (UID: \"58ec3eb9-233e-470e-82bc-7016926a5acc\") " pod="kuadrant-system/authorino-76877994-7tdb9" Jul 07 17:50:28.665751 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.665579 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55tdz\" (UniqueName: \"kubernetes.io/projected/58ec3eb9-233e-470e-82bc-7016926a5acc-kube-api-access-55tdz\") pod \"authorino-76877994-7tdb9\" (UID: \"58ec3eb9-233e-470e-82bc-7016926a5acc\") " pod="kuadrant-system/authorino-76877994-7tdb9" Jul 07 17:50:28.685231 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.685188 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-7tdb9"] Jul 07 17:50:28.685522 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:50:28.685488 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-55tdz tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-7tdb9" podUID="58ec3eb9-233e-470e-82bc-7016926a5acc" Jul 07 17:50:28.766469 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.766426 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58ec3eb9-233e-470e-82bc-7016926a5acc-tls-cert\") pod \"authorino-76877994-7tdb9\" (UID: \"58ec3eb9-233e-470e-82bc-7016926a5acc\") " pod="kuadrant-system/authorino-76877994-7tdb9" Jul 07 17:50:28.766696 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.766510 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55tdz\" (UniqueName: \"kubernetes.io/projected/58ec3eb9-233e-470e-82bc-7016926a5acc-kube-api-access-55tdz\") pod \"authorino-76877994-7tdb9\" (UID: \"58ec3eb9-233e-470e-82bc-7016926a5acc\") " pod="kuadrant-system/authorino-76877994-7tdb9" Jul 07 17:50:28.769091 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.769066 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58ec3eb9-233e-470e-82bc-7016926a5acc-tls-cert\") pod \"authorino-76877994-7tdb9\" (UID: \"58ec3eb9-233e-470e-82bc-7016926a5acc\") " pod="kuadrant-system/authorino-76877994-7tdb9" Jul 07 17:50:28.799803 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:28.799772 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55tdz\" (UniqueName: \"kubernetes.io/projected/58ec3eb9-233e-470e-82bc-7016926a5acc-kube-api-access-55tdz\") pod \"authorino-76877994-7tdb9\" (UID: \"58ec3eb9-233e-470e-82bc-7016926a5acc\") " pod="kuadrant-system/authorino-76877994-7tdb9" Jul 07 17:50:29.538523 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:29.538482 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-7tdb9" Jul 07 17:50:29.539000 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:29.538486 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-9rbtg" Jul 07 17:50:29.549646 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:29.549618 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-7tdb9" Jul 07 17:50:29.552510 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:29.552485 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-9rbtg" Jul 07 17:50:29.572329 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:29.572294 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58ec3eb9-233e-470e-82bc-7016926a5acc-tls-cert\") pod \"58ec3eb9-233e-470e-82bc-7016926a5acc\" (UID: \"58ec3eb9-233e-470e-82bc-7016926a5acc\") " Jul 07 17:50:29.572522 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:29.572345 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mqpm\" (UniqueName: \"kubernetes.io/projected/d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9-kube-api-access-2mqpm\") pod \"d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9\" (UID: \"d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9\") " Jul 07 17:50:29.572522 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:29.572388 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55tdz\" (UniqueName: \"kubernetes.io/projected/58ec3eb9-233e-470e-82bc-7016926a5acc-kube-api-access-55tdz\") pod \"58ec3eb9-233e-470e-82bc-7016926a5acc\" (UID: \"58ec3eb9-233e-470e-82bc-7016926a5acc\") " Jul 07 17:50:29.574805 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:29.574770 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9-kube-api-access-2mqpm" (OuterVolumeSpecName: "kube-api-access-2mqpm") pod "d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9" (UID: "d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9"). InnerVolumeSpecName "kube-api-access-2mqpm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:50:29.575005 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:29.574815 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58ec3eb9-233e-470e-82bc-7016926a5acc-kube-api-access-55tdz" (OuterVolumeSpecName: "kube-api-access-55tdz") pod "58ec3eb9-233e-470e-82bc-7016926a5acc" (UID: "58ec3eb9-233e-470e-82bc-7016926a5acc"). InnerVolumeSpecName "kube-api-access-55tdz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:50:29.575005 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:29.574813 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58ec3eb9-233e-470e-82bc-7016926a5acc-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "58ec3eb9-233e-470e-82bc-7016926a5acc" (UID: "58ec3eb9-233e-470e-82bc-7016926a5acc"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:50:29.672890 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:29.672847 2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58ec3eb9-233e-470e-82bc-7016926a5acc-tls-cert\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:50:29.672890 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:29.672881 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mqpm\" (UniqueName: \"kubernetes.io/projected/d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9-kube-api-access-2mqpm\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:50:29.672890 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:29.672892 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55tdz\" (UniqueName: \"kubernetes.io/projected/58ec3eb9-233e-470e-82bc-7016926a5acc-kube-api-access-55tdz\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:50:30.543482 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:30.543450 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-7tdb9" Jul 07 17:50:30.543906 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:30.543450 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-9rbtg" Jul 07 17:50:30.577142 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:30.577104 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-7tdb9"] Jul 07 17:50:30.584396 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:30.584353 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-7tdb9"] Jul 07 17:50:30.607888 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:30.607844 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9rbtg"] Jul 07 17:50:30.609757 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:30.609730 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9rbtg"] Jul 07 17:50:32.516478 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:32.516439 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58ec3eb9-233e-470e-82bc-7016926a5acc" path="/var/lib/kubelet/pods/58ec3eb9-233e-470e-82bc-7016926a5acc/volumes" Jul 07 17:50:32.516868 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:32.516658 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9" path="/var/lib/kubelet/pods/d1bddfe1-0a72-4e25-a44c-45c3d6a6f3b9/volumes" Jul 07 17:50:32.609713 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:32.609664 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t69jl"] Jul 07 17:50:32.609946 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:32.609885 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-t69jl" podUID="e25b0658-7739-4952-8f2d-6c2a3b893de7" containerName="authorino" containerID="cri-o://df14b0b08d97de97c83cb6c231a60d34c3f5eab7efc77ac1fda5c509193c8606" gracePeriod=30 Jul 07 17:50:32.866212 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:32.866183 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t69jl" Jul 07 17:50:32.901233 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:32.901199 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9jnx\" (UniqueName: \"kubernetes.io/projected/e25b0658-7739-4952-8f2d-6c2a3b893de7-kube-api-access-t9jnx\") pod \"e25b0658-7739-4952-8f2d-6c2a3b893de7\" (UID: \"e25b0658-7739-4952-8f2d-6c2a3b893de7\") " Jul 07 17:50:32.903732 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:32.903702 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e25b0658-7739-4952-8f2d-6c2a3b893de7-kube-api-access-t9jnx" (OuterVolumeSpecName: "kube-api-access-t9jnx") pod "e25b0658-7739-4952-8f2d-6c2a3b893de7" (UID: "e25b0658-7739-4952-8f2d-6c2a3b893de7"). InnerVolumeSpecName "kube-api-access-t9jnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:50:33.002056 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:33.002010 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9jnx\" (UniqueName: \"kubernetes.io/projected/e25b0658-7739-4952-8f2d-6c2a3b893de7-kube-api-access-t9jnx\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:50:33.566092 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:33.566052 2577 generic.go:356] "Generic (PLEG): container finished" podID="e25b0658-7739-4952-8f2d-6c2a3b893de7" containerID="df14b0b08d97de97c83cb6c231a60d34c3f5eab7efc77ac1fda5c509193c8606" exitCode=0 Jul 07 17:50:33.566092 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:33.566098 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t69jl" Jul 07 17:50:33.566603 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:33.566135 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t69jl" event={"ID":"e25b0658-7739-4952-8f2d-6c2a3b893de7","Type":"ContainerDied","Data":"df14b0b08d97de97c83cb6c231a60d34c3f5eab7efc77ac1fda5c509193c8606"} Jul 07 17:50:33.566603 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:33.566182 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t69jl" event={"ID":"e25b0658-7739-4952-8f2d-6c2a3b893de7","Type":"ContainerDied","Data":"7bbb8e2c3d3c3736b12fed91678d8d134b6aec446c10932e7bdcf1dd41313798"} Jul 07 17:50:33.566603 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:33.566199 2577 scope.go:117] "RemoveContainer" containerID="df14b0b08d97de97c83cb6c231a60d34c3f5eab7efc77ac1fda5c509193c8606" Jul 07 17:50:33.580719 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:33.580685 2577 scope.go:117] "RemoveContainer" containerID="df14b0b08d97de97c83cb6c231a60d34c3f5eab7efc77ac1fda5c509193c8606" Jul 07 17:50:33.581544 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:50:33.581508 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df14b0b08d97de97c83cb6c231a60d34c3f5eab7efc77ac1fda5c509193c8606\": container with ID starting with df14b0b08d97de97c83cb6c231a60d34c3f5eab7efc77ac1fda5c509193c8606 not found: ID does not exist" containerID="df14b0b08d97de97c83cb6c231a60d34c3f5eab7efc77ac1fda5c509193c8606" Jul 07 17:50:33.581664 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:33.581556 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df14b0b08d97de97c83cb6c231a60d34c3f5eab7efc77ac1fda5c509193c8606"} err="failed to get container status \"df14b0b08d97de97c83cb6c231a60d34c3f5eab7efc77ac1fda5c509193c8606\": rpc error: code = NotFound desc = could not find container \"df14b0b08d97de97c83cb6c231a60d34c3f5eab7efc77ac1fda5c509193c8606\": container with ID starting with df14b0b08d97de97c83cb6c231a60d34c3f5eab7efc77ac1fda5c509193c8606 not found: ID does not exist" Jul 07 17:50:33.592804 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:33.592763 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t69jl"] Jul 07 17:50:33.598206 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:33.598171 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t69jl"] Jul 07 17:50:34.516203 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:34.516162 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e25b0658-7739-4952-8f2d-6c2a3b893de7" path="/var/lib/kubelet/pods/e25b0658-7739-4952-8f2d-6c2a3b893de7/volumes" Jul 07 17:50:37.186604 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.186564 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64588b5c89-g8k54"] Jul 07 17:50:37.186990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.186887 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e25b0658-7739-4952-8f2d-6c2a3b893de7" containerName="authorino" Jul 07 17:50:37.186990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.186897 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e25b0658-7739-4952-8f2d-6c2a3b893de7" containerName="authorino" Jul 07 17:50:37.186990 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.186950 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e25b0658-7739-4952-8f2d-6c2a3b893de7" containerName="authorino" Jul 07 17:50:37.200068 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.199999 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64588b5c89-g8k54" Jul 07 17:50:37.201460 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.201429 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64588b5c89-g8k54"] Jul 07 17:50:37.202677 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.202653 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 17:50:37.203853 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.203829 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2xbz2\"" Jul 07 17:50:37.203980 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.203881 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 17:50:37.238517 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.238470 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzwkf\" (UniqueName: \"kubernetes.io/projected/5216d63d-2a78-47a9-b340-8b2656319ad1-kube-api-access-mzwkf\") pod \"maas-controller-64588b5c89-g8k54\" (UID: \"5216d63d-2a78-47a9-b340-8b2656319ad1\") " pod="opendatahub/maas-controller-64588b5c89-g8k54" Jul 07 17:50:37.238727 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.238561 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5216d63d-2a78-47a9-b340-8b2656319ad1-webhook-cert\") pod \"maas-controller-64588b5c89-g8k54\" (UID: \"5216d63d-2a78-47a9-b340-8b2656319ad1\") " pod="opendatahub/maas-controller-64588b5c89-g8k54" Jul 07 17:50:37.339153 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.339109 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5216d63d-2a78-47a9-b340-8b2656319ad1-webhook-cert\") pod \"maas-controller-64588b5c89-g8k54\" (UID: \"5216d63d-2a78-47a9-b340-8b2656319ad1\") " pod="opendatahub/maas-controller-64588b5c89-g8k54" Jul 07 17:50:37.339372 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.339194 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzwkf\" (UniqueName: \"kubernetes.io/projected/5216d63d-2a78-47a9-b340-8b2656319ad1-kube-api-access-mzwkf\") pod \"maas-controller-64588b5c89-g8k54\" (UID: \"5216d63d-2a78-47a9-b340-8b2656319ad1\") " pod="opendatahub/maas-controller-64588b5c89-g8k54" Jul 07 17:50:37.341793 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.341765 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5216d63d-2a78-47a9-b340-8b2656319ad1-webhook-cert\") pod \"maas-controller-64588b5c89-g8k54\" (UID: \"5216d63d-2a78-47a9-b340-8b2656319ad1\") " pod="opendatahub/maas-controller-64588b5c89-g8k54" Jul 07 17:50:37.350214 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.350184 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzwkf\" (UniqueName: \"kubernetes.io/projected/5216d63d-2a78-47a9-b340-8b2656319ad1-kube-api-access-mzwkf\") pod \"maas-controller-64588b5c89-g8k54\" (UID: \"5216d63d-2a78-47a9-b340-8b2656319ad1\") " pod="opendatahub/maas-controller-64588b5c89-g8k54" Jul 07 17:50:37.377630 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.377594 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76d699bd45-qvd76"] Jul 07 17:50:37.380736 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.380717 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d699bd45-qvd76" Jul 07 17:50:37.390938 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.390905 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76d699bd45-qvd76"] Jul 07 17:50:37.440437 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.440404 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62c617c1-42c3-4804-a753-de89bc812f90-webhook-cert\") pod \"maas-controller-76d699bd45-qvd76\" (UID: \"62c617c1-42c3-4804-a753-de89bc812f90\") " pod="opendatahub/maas-controller-76d699bd45-qvd76" Jul 07 17:50:37.440437 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.440438 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8fcx\" (UniqueName: \"kubernetes.io/projected/62c617c1-42c3-4804-a753-de89bc812f90-kube-api-access-q8fcx\") pod \"maas-controller-76d699bd45-qvd76\" (UID: \"62c617c1-42c3-4804-a753-de89bc812f90\") " pod="opendatahub/maas-controller-76d699bd45-qvd76" Jul 07 17:50:37.512775 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.512734 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64588b5c89-g8k54" Jul 07 17:50:37.541889 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.541851 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62c617c1-42c3-4804-a753-de89bc812f90-webhook-cert\") pod \"maas-controller-76d699bd45-qvd76\" (UID: \"62c617c1-42c3-4804-a753-de89bc812f90\") " pod="opendatahub/maas-controller-76d699bd45-qvd76" Jul 07 17:50:37.541889 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.541891 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8fcx\" (UniqueName: \"kubernetes.io/projected/62c617c1-42c3-4804-a753-de89bc812f90-kube-api-access-q8fcx\") pod \"maas-controller-76d699bd45-qvd76\" (UID: \"62c617c1-42c3-4804-a753-de89bc812f90\") " pod="opendatahub/maas-controller-76d699bd45-qvd76" Jul 07 17:50:37.544707 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.544675 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62c617c1-42c3-4804-a753-de89bc812f90-webhook-cert\") pod \"maas-controller-76d699bd45-qvd76\" (UID: \"62c617c1-42c3-4804-a753-de89bc812f90\") " pod="opendatahub/maas-controller-76d699bd45-qvd76" Jul 07 17:50:37.551845 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.551810 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8fcx\" (UniqueName: \"kubernetes.io/projected/62c617c1-42c3-4804-a753-de89bc812f90-kube-api-access-q8fcx\") pod \"maas-controller-76d699bd45-qvd76\" (UID: \"62c617c1-42c3-4804-a753-de89bc812f90\") " pod="opendatahub/maas-controller-76d699bd45-qvd76" Jul 07 17:50:37.660457 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.660422 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64588b5c89-g8k54"] Jul 07 17:50:37.661701 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:50:37.661669 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5216d63d_2a78_47a9_b340_8b2656319ad1.slice/crio-7fb4c957166fed4e23f6b16c9dbec6c6a0eb707262f16fd52155280a53ff976d WatchSource:0}: Error finding container 7fb4c957166fed4e23f6b16c9dbec6c6a0eb707262f16fd52155280a53ff976d: Status 404 returned error can't find the container with id 7fb4c957166fed4e23f6b16c9dbec6c6a0eb707262f16fd52155280a53ff976d Jul 07 17:50:37.663133 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.663112 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 17:50:37.692211 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.692103 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d699bd45-qvd76" Jul 07 17:50:37.837939 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:37.837911 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76d699bd45-qvd76"] Jul 07 17:50:37.840575 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:50:37.840545 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c617c1_42c3_4804_a753_de89bc812f90.slice/crio-c32b9a3cba609f71fc05c7584bdd75d7f0110801c59b36e110212a973d1ad2a9 WatchSource:0}: Error finding container c32b9a3cba609f71fc05c7584bdd75d7f0110801c59b36e110212a973d1ad2a9: Status 404 returned error can't find the container with id c32b9a3cba609f71fc05c7584bdd75d7f0110801c59b36e110212a973d1ad2a9 Jul 07 17:50:38.595564 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:38.595525 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d699bd45-qvd76" event={"ID":"62c617c1-42c3-4804-a753-de89bc812f90","Type":"ContainerStarted","Data":"c32b9a3cba609f71fc05c7584bdd75d7f0110801c59b36e110212a973d1ad2a9"} Jul 07 17:50:38.597494 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:38.597454 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64588b5c89-g8k54" event={"ID":"5216d63d-2a78-47a9-b340-8b2656319ad1","Type":"ContainerStarted","Data":"7fb4c957166fed4e23f6b16c9dbec6c6a0eb707262f16fd52155280a53ff976d"} Jul 07 17:50:40.610622 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:40.610584 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64588b5c89-g8k54" event={"ID":"5216d63d-2a78-47a9-b340-8b2656319ad1","Type":"ContainerStarted","Data":"589f25a90e5e6a2c66c6d3199a4839a8d92545eda59f1c3dbce8d8fc5f761a65"} Jul 07 17:50:40.611172 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:40.610656 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64588b5c89-g8k54" Jul 07 17:50:40.612278 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:40.612247 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d699bd45-qvd76" event={"ID":"62c617c1-42c3-4804-a753-de89bc812f90","Type":"ContainerStarted","Data":"9dfbed3d7d2ec495390e9e64f51f5d0cb53770288b3bbd336759dda71d4fbb99"} Jul 07 17:50:40.612407 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:40.612392 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76d699bd45-qvd76" Jul 07 17:50:40.631915 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:40.631852 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64588b5c89-g8k54" podStartSLOduration=1.247431985 podStartE2EDuration="3.631832562s" totalImagesPullingTime="2.384400577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:50:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:50:40.630134604 +0000 UTC m=+644.669461507" watchObservedRunningTime="2026-07-07 17:50:40.631832562 +0000 UTC m=+644.671159464" Jul 07 17:50:40.653300 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:40.653238 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76d699bd45-qvd76" podStartSLOduration=1.436090637 podStartE2EDuration="3.653218598s" totalImagesPullingTime="2.217127961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:50:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:50:40.652093015 +0000 UTC m=+644.691419917" watchObservedRunningTime="2026-07-07 17:50:40.653218598 +0000 UTC m=+644.692545500" Jul 07 17:50:51.631133 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:51.631100 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64588b5c89-g8k54" Jul 07 17:50:51.631580 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:51.631181 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76d699bd45-qvd76" Jul 07 17:50:51.709404 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:51.709370 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64588b5c89-g8k54"] Jul 07 17:50:51.709600 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:51.709563 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-64588b5c89-g8k54" podUID="5216d63d-2a78-47a9-b340-8b2656319ad1" containerName="manager" containerID="cri-o://589f25a90e5e6a2c66c6d3199a4839a8d92545eda59f1c3dbce8d8fc5f761a65" gracePeriod=30 Jul 07 17:50:51.955985 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:51.955953 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64588b5c89-g8k54" Jul 07 17:50:52.074349 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.074306 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5216d63d-2a78-47a9-b340-8b2656319ad1-webhook-cert\") pod \"5216d63d-2a78-47a9-b340-8b2656319ad1\" (UID: \"5216d63d-2a78-47a9-b340-8b2656319ad1\") " Jul 07 17:50:52.074349 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.074353 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzwkf\" (UniqueName: \"kubernetes.io/projected/5216d63d-2a78-47a9-b340-8b2656319ad1-kube-api-access-mzwkf\") pod \"5216d63d-2a78-47a9-b340-8b2656319ad1\" (UID: \"5216d63d-2a78-47a9-b340-8b2656319ad1\") " Jul 07 17:50:52.076935 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.076900 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5216d63d-2a78-47a9-b340-8b2656319ad1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5216d63d-2a78-47a9-b340-8b2656319ad1" (UID: "5216d63d-2a78-47a9-b340-8b2656319ad1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:50:52.076935 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.076900 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5216d63d-2a78-47a9-b340-8b2656319ad1-kube-api-access-mzwkf" (OuterVolumeSpecName: "kube-api-access-mzwkf") pod "5216d63d-2a78-47a9-b340-8b2656319ad1" (UID: "5216d63d-2a78-47a9-b340-8b2656319ad1"). InnerVolumeSpecName "kube-api-access-mzwkf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:50:52.175515 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.175476 2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5216d63d-2a78-47a9-b340-8b2656319ad1-webhook-cert\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:50:52.175515 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.175508 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzwkf\" (UniqueName: \"kubernetes.io/projected/5216d63d-2a78-47a9-b340-8b2656319ad1-kube-api-access-mzwkf\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:50:52.681644 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.681606 2577 generic.go:356] "Generic (PLEG): container finished" podID="5216d63d-2a78-47a9-b340-8b2656319ad1" containerID="589f25a90e5e6a2c66c6d3199a4839a8d92545eda59f1c3dbce8d8fc5f761a65" exitCode=0 Jul 07 17:50:52.682211 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.681654 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64588b5c89-g8k54" Jul 07 17:50:52.682211 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.681700 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64588b5c89-g8k54" event={"ID":"5216d63d-2a78-47a9-b340-8b2656319ad1","Type":"ContainerDied","Data":"589f25a90e5e6a2c66c6d3199a4839a8d92545eda59f1c3dbce8d8fc5f761a65"} Jul 07 17:50:52.682211 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.681772 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64588b5c89-g8k54" event={"ID":"5216d63d-2a78-47a9-b340-8b2656319ad1","Type":"ContainerDied","Data":"7fb4c957166fed4e23f6b16c9dbec6c6a0eb707262f16fd52155280a53ff976d"} Jul 07 17:50:52.682211 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.681805 2577 scope.go:117] "RemoveContainer" containerID="589f25a90e5e6a2c66c6d3199a4839a8d92545eda59f1c3dbce8d8fc5f761a65" Jul 07 17:50:52.691431 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.691410 2577 scope.go:117] "RemoveContainer" containerID="589f25a90e5e6a2c66c6d3199a4839a8d92545eda59f1c3dbce8d8fc5f761a65" Jul 07 17:50:52.691756 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:50:52.691735 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"589f25a90e5e6a2c66c6d3199a4839a8d92545eda59f1c3dbce8d8fc5f761a65\": container with ID starting with 589f25a90e5e6a2c66c6d3199a4839a8d92545eda59f1c3dbce8d8fc5f761a65 not found: ID does not exist" containerID="589f25a90e5e6a2c66c6d3199a4839a8d92545eda59f1c3dbce8d8fc5f761a65" Jul 07 17:50:52.691823 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.691771 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"589f25a90e5e6a2c66c6d3199a4839a8d92545eda59f1c3dbce8d8fc5f761a65"} err="failed to get container status \"589f25a90e5e6a2c66c6d3199a4839a8d92545eda59f1c3dbce8d8fc5f761a65\": rpc error: code = NotFound desc = could not find container \"589f25a90e5e6a2c66c6d3199a4839a8d92545eda59f1c3dbce8d8fc5f761a65\": container with ID starting with 589f25a90e5e6a2c66c6d3199a4839a8d92545eda59f1c3dbce8d8fc5f761a65 not found: ID does not exist" Jul 07 17:50:52.699645 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.699601 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64588b5c89-g8k54"] Jul 07 17:50:52.703433 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:52.703396 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64588b5c89-g8k54"] Jul 07 17:50:54.516061 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:50:54.516018 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5216d63d-2a78-47a9-b340-8b2656319ad1" path="/var/lib/kubelet/pods/5216d63d-2a78-47a9-b340-8b2656319ad1/volumes" Jul 07 17:51:12.238227 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.238188 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-65f5bb9c48-82jmb"] Jul 07 17:51:12.238695 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.238545 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5216d63d-2a78-47a9-b340-8b2656319ad1" containerName="manager" Jul 07 17:51:12.238695 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.238557 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5216d63d-2a78-47a9-b340-8b2656319ad1" containerName="manager" Jul 07 17:51:12.238695 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.238622 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5216d63d-2a78-47a9-b340-8b2656319ad1" containerName="manager" Jul 07 17:51:12.244581 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.244555 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-65f5bb9c48-82jmb" Jul 07 17:51:12.247248 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.247223 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 07 17:51:12.247248 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.247238 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-bmbdf\"" Jul 07 17:51:12.253711 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.253684 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-65f5bb9c48-82jmb"] Jul 07 17:51:12.258355 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.258319 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0355921f-86fb-4ad0-ba9a-ead9a72baca8-openshift-service-ca\") pod \"maas-api-65f5bb9c48-82jmb\" (UID: \"0355921f-86fb-4ad0-ba9a-ead9a72baca8\") " pod="opendatahub/maas-api-65f5bb9c48-82jmb" Jul 07 17:51:12.258543 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.258380 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0355921f-86fb-4ad0-ba9a-ead9a72baca8-maas-api-tls\") pod \"maas-api-65f5bb9c48-82jmb\" (UID: \"0355921f-86fb-4ad0-ba9a-ead9a72baca8\") " pod="opendatahub/maas-api-65f5bb9c48-82jmb" Jul 07 17:51:12.258543 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.258407 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2cx5\" (UniqueName: \"kubernetes.io/projected/0355921f-86fb-4ad0-ba9a-ead9a72baca8-kube-api-access-k2cx5\") pod \"maas-api-65f5bb9c48-82jmb\" (UID: \"0355921f-86fb-4ad0-ba9a-ead9a72baca8\") " pod="opendatahub/maas-api-65f5bb9c48-82jmb" Jul 07 17:51:12.359369 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.359327 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2cx5\" (UniqueName: \"kubernetes.io/projected/0355921f-86fb-4ad0-ba9a-ead9a72baca8-kube-api-access-k2cx5\") pod \"maas-api-65f5bb9c48-82jmb\" (UID: \"0355921f-86fb-4ad0-ba9a-ead9a72baca8\") " pod="opendatahub/maas-api-65f5bb9c48-82jmb" Jul 07 17:51:12.359562 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.359428 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0355921f-86fb-4ad0-ba9a-ead9a72baca8-openshift-service-ca\") pod \"maas-api-65f5bb9c48-82jmb\" (UID: \"0355921f-86fb-4ad0-ba9a-ead9a72baca8\") " pod="opendatahub/maas-api-65f5bb9c48-82jmb" Jul 07 17:51:12.359562 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.359501 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0355921f-86fb-4ad0-ba9a-ead9a72baca8-maas-api-tls\") pod \"maas-api-65f5bb9c48-82jmb\" (UID: \"0355921f-86fb-4ad0-ba9a-ead9a72baca8\") " pod="opendatahub/maas-api-65f5bb9c48-82jmb" Jul 07 17:51:12.359648 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:51:12.359594 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 07 17:51:12.359687 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:51:12.359671 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0355921f-86fb-4ad0-ba9a-ead9a72baca8-maas-api-tls podName:0355921f-86fb-4ad0-ba9a-ead9a72baca8 nodeName:}" failed. No retries permitted until 2026-07-07 17:51:12.859650225 +0000 UTC m=+676.898977106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0355921f-86fb-4ad0-ba9a-ead9a72baca8-maas-api-tls") pod "maas-api-65f5bb9c48-82jmb" (UID: "0355921f-86fb-4ad0-ba9a-ead9a72baca8") : secret "maas-api-serving-cert" not found Jul 07 17:51:12.360149 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.360131 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0355921f-86fb-4ad0-ba9a-ead9a72baca8-openshift-service-ca\") pod \"maas-api-65f5bb9c48-82jmb\" (UID: \"0355921f-86fb-4ad0-ba9a-ead9a72baca8\") " pod="opendatahub/maas-api-65f5bb9c48-82jmb" Jul 07 17:51:12.369119 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.369087 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2cx5\" (UniqueName: \"kubernetes.io/projected/0355921f-86fb-4ad0-ba9a-ead9a72baca8-kube-api-access-k2cx5\") pod \"maas-api-65f5bb9c48-82jmb\" (UID: \"0355921f-86fb-4ad0-ba9a-ead9a72baca8\") " pod="opendatahub/maas-api-65f5bb9c48-82jmb" Jul 07 17:51:12.796902 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.796866 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-hrxjc"] Jul 07 17:51:12.801553 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.801526 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-hrxjc" Jul 07 17:51:12.805105 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.805076 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 17:51:12.805963 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.805941 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-d42tv\"" Jul 07 17:51:12.814411 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.814375 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-hrxjc"] Jul 07 17:51:12.840693 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.840652 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-lh77w"] Jul 07 17:51:12.845213 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.845187 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-lh77w" Jul 07 17:51:12.855623 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.855592 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-lh77w"] Jul 07 17:51:12.862315 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.862280 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc-plugins-config-volume\") pod \"payload-processing-58f86c7798-hrxjc\" (UID: \"cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc\") " pod="openshift-ingress/payload-processing-58f86c7798-hrxjc" Jul 07 17:51:12.862524 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.862336 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0355921f-86fb-4ad0-ba9a-ead9a72baca8-maas-api-tls\") pod \"maas-api-65f5bb9c48-82jmb\" (UID: \"0355921f-86fb-4ad0-ba9a-ead9a72baca8\") " pod="opendatahub/maas-api-65f5bb9c48-82jmb" Jul 07 17:51:12.862524 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.862366 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxhdt\" (UniqueName: \"kubernetes.io/projected/d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe-kube-api-access-vxhdt\") pod \"payload-pre-processing-7964457696-lh77w\" (UID: \"d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe\") " pod="openshift-ingress/payload-pre-processing-7964457696-lh77w" Jul 07 17:51:12.862524 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.862405 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe-plugins-config-volume\") pod \"payload-pre-processing-7964457696-lh77w\" (UID: \"d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe\") " pod="openshift-ingress/payload-pre-processing-7964457696-lh77w" Jul 07 17:51:12.862524 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.862441 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6q46\" (UniqueName: \"kubernetes.io/projected/cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc-kube-api-access-f6q46\") pod \"payload-processing-58f86c7798-hrxjc\" (UID: \"cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc\") " pod="openshift-ingress/payload-processing-58f86c7798-hrxjc" Jul 07 17:51:12.865626 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.865596 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0355921f-86fb-4ad0-ba9a-ead9a72baca8-maas-api-tls\") pod \"maas-api-65f5bb9c48-82jmb\" (UID: \"0355921f-86fb-4ad0-ba9a-ead9a72baca8\") " pod="opendatahub/maas-api-65f5bb9c48-82jmb" Jul 07 17:51:12.963616 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.963575 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxhdt\" (UniqueName: \"kubernetes.io/projected/d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe-kube-api-access-vxhdt\") pod \"payload-pre-processing-7964457696-lh77w\" (UID: \"d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe\") " pod="openshift-ingress/payload-pre-processing-7964457696-lh77w" Jul 07 17:51:12.963833 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.963633 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe-plugins-config-volume\") pod \"payload-pre-processing-7964457696-lh77w\" (UID: \"d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe\") " pod="openshift-ingress/payload-pre-processing-7964457696-lh77w" Jul 07 17:51:12.963833 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.963672 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6q46\" (UniqueName: \"kubernetes.io/projected/cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc-kube-api-access-f6q46\") pod \"payload-processing-58f86c7798-hrxjc\" (UID: \"cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc\") " pod="openshift-ingress/payload-processing-58f86c7798-hrxjc" Jul 07 17:51:12.963833 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.963697 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc-plugins-config-volume\") pod \"payload-processing-58f86c7798-hrxjc\" (UID: \"cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc\") " pod="openshift-ingress/payload-processing-58f86c7798-hrxjc" Jul 07 17:51:12.964493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.964472 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc-plugins-config-volume\") pod \"payload-processing-58f86c7798-hrxjc\" (UID: \"cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc\") " pod="openshift-ingress/payload-processing-58f86c7798-hrxjc" Jul 07 17:51:12.964558 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.964475 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe-plugins-config-volume\") pod \"payload-pre-processing-7964457696-lh77w\" (UID: \"d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe\") " pod="openshift-ingress/payload-pre-processing-7964457696-lh77w" Jul 07 17:51:12.972287 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.972232 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6q46\" (UniqueName: \"kubernetes.io/projected/cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc-kube-api-access-f6q46\") pod \"payload-processing-58f86c7798-hrxjc\" (UID: \"cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc\") " pod="openshift-ingress/payload-processing-58f86c7798-hrxjc" Jul 07 17:51:12.972474 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:12.972252 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxhdt\" (UniqueName: \"kubernetes.io/projected/d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe-kube-api-access-vxhdt\") pod \"payload-pre-processing-7964457696-lh77w\" (UID: \"d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe\") " pod="openshift-ingress/payload-pre-processing-7964457696-lh77w" Jul 07 17:51:13.113619 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:13.113516 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-hrxjc" Jul 07 17:51:13.156580 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:13.156547 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-65f5bb9c48-82jmb" Jul 07 17:51:13.157610 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:13.157576 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-lh77w" Jul 07 17:51:13.299409 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:13.298460 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-hrxjc"] Jul 07 17:51:13.302158 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:51:13.302108 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbe88c7_f58d_4e0f_bae2_bd9d95cbd9dc.slice/crio-264a36f140a7b1f2383bda9cf07e00055dbcb24c2917e39770f4a6e0a873f005 WatchSource:0}: Error finding container 264a36f140a7b1f2383bda9cf07e00055dbcb24c2917e39770f4a6e0a873f005: Status 404 returned error can't find the container with id 264a36f140a7b1f2383bda9cf07e00055dbcb24c2917e39770f4a6e0a873f005 Jul 07 17:51:13.343772 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:13.343697 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-65f5bb9c48-82jmb"] Jul 07 17:51:13.346821 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:51:13.346785 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0355921f_86fb_4ad0_ba9a_ead9a72baca8.slice/crio-03094d016d409d00b09648795a97d587f0375bb892158a2b64f78c6ceab1a7c6 WatchSource:0}: Error finding container 03094d016d409d00b09648795a97d587f0375bb892158a2b64f78c6ceab1a7c6: Status 404 returned error can't find the container with id 03094d016d409d00b09648795a97d587f0375bb892158a2b64f78c6ceab1a7c6 Jul 07 17:51:13.368709 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:13.368683 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-lh77w"] Jul 07 17:51:13.370931 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:51:13.370900 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b3ea4b_48aa_4aab_9397_2024cb0f4bbe.slice/crio-b3c5726fde872617d974f3b38ef61a64ebda12683b2cfa29a77fe0da245167a6 WatchSource:0}: Error finding container b3c5726fde872617d974f3b38ef61a64ebda12683b2cfa29a77fe0da245167a6: Status 404 returned error can't find the container with id b3c5726fde872617d974f3b38ef61a64ebda12683b2cfa29a77fe0da245167a6 Jul 07 17:51:13.793375 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:13.793333 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-lh77w" event={"ID":"d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe","Type":"ContainerStarted","Data":"b3c5726fde872617d974f3b38ef61a64ebda12683b2cfa29a77fe0da245167a6"} Jul 07 17:51:13.794677 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:13.794643 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-65f5bb9c48-82jmb" event={"ID":"0355921f-86fb-4ad0-ba9a-ead9a72baca8","Type":"ContainerStarted","Data":"03094d016d409d00b09648795a97d587f0375bb892158a2b64f78c6ceab1a7c6"} Jul 07 17:51:13.795882 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:13.795852 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-hrxjc" event={"ID":"cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc","Type":"ContainerStarted","Data":"264a36f140a7b1f2383bda9cf07e00055dbcb24c2917e39770f4a6e0a873f005"} Jul 07 17:51:14.805243 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:14.805200 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-hrxjc" event={"ID":"cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc","Type":"ContainerStarted","Data":"27237d68c764c091fe02d472c74b9630f09e620b4d267cdf34508b2fe31bbac6"} Jul 07 17:51:14.805733 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:14.805284 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-hrxjc" Jul 07 17:51:14.807725 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:14.807472 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-lh77w" event={"ID":"d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe","Type":"ContainerStarted","Data":"137036eef4e42de4e315d87e612eb8442ed581aa003c6c1b63e58193978c9223"} Jul 07 17:51:14.807725 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:14.807629 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-lh77w" Jul 07 17:51:14.820418 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:14.820359 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-hrxjc" podStartSLOduration=1.470294429 podStartE2EDuration="2.820339462s" totalImagesPullingTime="1.350045033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:51:14.819303804 +0000 UTC m=+678.858630706" watchObservedRunningTime="2026-07-07 17:51:14.820339462 +0000 UTC m=+678.859666366" Jul 07 17:51:14.836459 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:14.836390 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-lh77w" podStartSLOduration=1.55244741 podStartE2EDuration="2.836369235s" totalImagesPullingTime="1.283921825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:51:14.834128397 +0000 UTC m=+678.873455301" watchObservedRunningTime="2026-07-07 17:51:14.836369235 +0000 UTC m=+678.875696137" Jul 07 17:51:15.814633 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:15.814586 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-65f5bb9c48-82jmb" event={"ID":"0355921f-86fb-4ad0-ba9a-ead9a72baca8","Type":"ContainerStarted","Data":"86a37a7e5dc503735c90bd1079011d44a47c8621dc053187425dd5f4e5969dbf"} Jul 07 17:51:15.815164 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:15.815013 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-65f5bb9c48-82jmb" Jul 07 17:51:15.832360 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:15.832293 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-65f5bb9c48-82jmb" podStartSLOduration=1.995592786 podStartE2EDuration="3.832272219s" totalImagesPullingTime="1.836679433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:51:15.83168913 +0000 UTC m=+679.871016058" watchObservedRunningTime="2026-07-07 17:51:15.832272219 +0000 UTC m=+679.871599123" Jul 07 17:51:21.826534 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:21.826498 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-65f5bb9c48-82jmb" Jul 07 17:51:25.817070 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:25.817022 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-hrxjc" Jul 07 17:51:25.817564 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:25.817124 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-lh77w" Jul 07 17:51:59.003508 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.003420 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4"] Jul 07 17:51:59.007184 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.007150 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.009576 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.009551 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 17:51:59.010347 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.010315 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 17:51:59.010493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.010316 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 07 17:51:59.010493 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.010315 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xtlc6\"" Jul 07 17:51:59.019521 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.019490 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4"] Jul 07 17:51:59.177076 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.176996 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e5876b7-495a-40a5-b519-93560475aef7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.177076 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.177077 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e5876b7-495a-40a5-b519-93560475aef7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.177308 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.177100 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e5876b7-495a-40a5-b519-93560475aef7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.177308 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.177124 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e5876b7-495a-40a5-b519-93560475aef7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.177308 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.177218 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e5876b7-495a-40a5-b519-93560475aef7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.177308 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.177263 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd8jr\" (UniqueName: \"kubernetes.io/projected/9e5876b7-495a-40a5-b519-93560475aef7-kube-api-access-dd8jr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.278384 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.278281 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e5876b7-495a-40a5-b519-93560475aef7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.278384 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.278333 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd8jr\" (UniqueName: \"kubernetes.io/projected/9e5876b7-495a-40a5-b519-93560475aef7-kube-api-access-dd8jr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.278628 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.278467 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e5876b7-495a-40a5-b519-93560475aef7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.278628 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.278520 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e5876b7-495a-40a5-b519-93560475aef7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.278628 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.278554 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e5876b7-495a-40a5-b519-93560475aef7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.278628 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.278591 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e5876b7-495a-40a5-b519-93560475aef7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.278840 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.278743 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e5876b7-495a-40a5-b519-93560475aef7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.278937 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.278918 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e5876b7-495a-40a5-b519-93560475aef7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.278980 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.278928 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e5876b7-495a-40a5-b519-93560475aef7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.280945 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.280923 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e5876b7-495a-40a5-b519-93560475aef7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.281239 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.281220 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e5876b7-495a-40a5-b519-93560475aef7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.286941 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.286911 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd8jr\" (UniqueName: \"kubernetes.io/projected/9e5876b7-495a-40a5-b519-93560475aef7-kube-api-access-dd8jr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sn4b4\" (UID: \"9e5876b7-495a-40a5-b519-93560475aef7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.319623 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.319578 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:51:59.471919 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:51:59.471890 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4"] Jul 07 17:51:59.474006 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:51:59.473976 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e5876b7_495a_40a5_b519_93560475aef7.slice/crio-0b7b5b9e711e428fc416f301b7229ae84ceffa59348aa72814fbc864441d38be WatchSource:0}: Error finding container 0b7b5b9e711e428fc416f301b7229ae84ceffa59348aa72814fbc864441d38be: Status 404 returned error can't find the container with id 0b7b5b9e711e428fc416f301b7229ae84ceffa59348aa72814fbc864441d38be Jul 07 17:52:00.056021 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:00.055979 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" event={"ID":"9e5876b7-495a-40a5-b519-93560475aef7","Type":"ContainerStarted","Data":"0b7b5b9e711e428fc416f301b7229ae84ceffa59348aa72814fbc864441d38be"} Jul 07 17:52:01.065675 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:01.065639 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" event={"ID":"9e5876b7-495a-40a5-b519-93560475aef7","Type":"ContainerStarted","Data":"182429f709dac95f08187995085fbcef5fc18ae0159ef9f08bb653650d36798b"} Jul 07 17:52:01.085020 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:01.084963 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" podStartSLOduration=2.047599852 podStartE2EDuration="3.084946349s" totalImagesPullingTime="1.037346497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:52:01.084318475 +0000 UTC m=+725.123645403" watchObservedRunningTime="2026-07-07 17:52:01.084946349 +0000 UTC m=+725.124273251" Jul 07 17:52:04.890691 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:04.890647 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k"] Jul 07 17:52:04.894519 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:04.894496 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:04.897241 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:04.897210 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 07 17:52:04.904222 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:04.904181 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k"] Jul 07 17:52:04.931868 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:04.931828 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:04.932103 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:04.931893 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kfhd\" (UniqueName: \"kubernetes.io/projected/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-kube-api-access-5kfhd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:04.932103 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:04.931946 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:04.932103 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:04.931978 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:04.932103 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:04.931997 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:04.932103 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:04.932026 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:05.032805 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:05.032757 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:05.033014 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:05.032819 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:05.033014 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:05.032869 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:05.033014 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:05.032932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kfhd\" (UniqueName: \"kubernetes.io/projected/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-kube-api-access-5kfhd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:05.033014 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:05.032983 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:05.033014 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:05.033010 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:05.033338 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:05.033303 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:05.033415 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:05.033393 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:05.033464 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:05.033449 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:05.035540 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:05.035507 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:05.035676 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:05.035659 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:05.040729 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:05.040701 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kfhd\" (UniqueName: \"kubernetes.io/projected/fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9-kube-api-access-5kfhd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-98w2k\" (UID: \"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:05.206832 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:05.206789 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:05.354490 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:05.354463 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k"] Jul 07 17:52:05.356558 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:52:05.356524 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcbb157c_2999_4ec8_bb4c_d6b8b0c8a2b9.slice/crio-932ed53a299654b2d3a077ef9608fcde4345fc86ed2d2715dc220e0023576077 WatchSource:0}: Error finding container 932ed53a299654b2d3a077ef9608fcde4345fc86ed2d2715dc220e0023576077: Status 404 returned error can't find the container with id 932ed53a299654b2d3a077ef9608fcde4345fc86ed2d2715dc220e0023576077 Jul 07 17:52:06.094544 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:06.094499 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" event={"ID":"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9","Type":"ContainerStarted","Data":"35c7dd144c6ff234625d2b726702d82a1ae339326d50bf6a3eb917230895309d"} Jul 07 17:52:06.094967 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:06.094570 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" event={"ID":"fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9","Type":"ContainerStarted","Data":"932ed53a299654b2d3a077ef9608fcde4345fc86ed2d2715dc220e0023576077"} Jul 07 17:52:06.113105 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:06.113052 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" podStartSLOduration=1.8982519500000001 podStartE2EDuration="2.113021771s" totalImagesPullingTime="214.769821ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:52:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:52:06.110632031 +0000 UTC m=+730.149958936" watchObservedRunningTime="2026-07-07 17:52:06.113021771 +0000 UTC m=+730.152348674" Jul 07 17:52:09.320602 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:09.320564 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:52:09.321034 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:09.320645 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:52:09.333460 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:09.333427 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:52:10.127833 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:10.127805 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sn4b4" Jul 07 17:52:15.207541 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:15.207500 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:15.208143 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:15.207565 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:15.220396 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:15.220364 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:16.158881 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:16.158848 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-98w2k" Jul 07 17:52:29.991420 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:29.991381 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l"] Jul 07 17:52:30.000605 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.000569 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.003835 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.003801 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 07 17:52:30.008129 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.008098 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l"] Jul 07 17:52:30.058675 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.058644 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61d2198e-3245-473d-9ee0-33e02270e9b0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.058675 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.058681 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61d2198e-3245-473d-9ee0-33e02270e9b0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.058895 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.058728 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhwrm\" (UniqueName: \"kubernetes.io/projected/61d2198e-3245-473d-9ee0-33e02270e9b0-kube-api-access-rhwrm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.058895 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.058810 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61d2198e-3245-473d-9ee0-33e02270e9b0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.058895 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.058845 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61d2198e-3245-473d-9ee0-33e02270e9b0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.058895 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.058880 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61d2198e-3245-473d-9ee0-33e02270e9b0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.159945 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.159898 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhwrm\" (UniqueName: \"kubernetes.io/projected/61d2198e-3245-473d-9ee0-33e02270e9b0-kube-api-access-rhwrm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.160183 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.159959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61d2198e-3245-473d-9ee0-33e02270e9b0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.160183 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.159980 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61d2198e-3245-473d-9ee0-33e02270e9b0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.160183 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.160011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61d2198e-3245-473d-9ee0-33e02270e9b0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.160183 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.160111 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61d2198e-3245-473d-9ee0-33e02270e9b0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.160183 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.160128 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61d2198e-3245-473d-9ee0-33e02270e9b0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.160513 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.160486 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61d2198e-3245-473d-9ee0-33e02270e9b0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.160513 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.160500 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61d2198e-3245-473d-9ee0-33e02270e9b0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.160612 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.160587 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61d2198e-3245-473d-9ee0-33e02270e9b0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.162683 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.162654 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61d2198e-3245-473d-9ee0-33e02270e9b0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.162927 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.162907 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61d2198e-3245-473d-9ee0-33e02270e9b0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.169019 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.168969 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhwrm\" (UniqueName: \"kubernetes.io/projected/61d2198e-3245-473d-9ee0-33e02270e9b0-kube-api-access-rhwrm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l\" (UID: \"61d2198e-3245-473d-9ee0-33e02270e9b0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.314725 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.314627 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:30.468467 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:30.468440 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l"] Jul 07 17:52:30.470621 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:52:30.470592 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61d2198e_3245_473d_9ee0_33e02270e9b0.slice/crio-aba843a1ed642a52ecf018eabf0d421bdd7d1ae11333b2afaa1f103d874b449d WatchSource:0}: Error finding container aba843a1ed642a52ecf018eabf0d421bdd7d1ae11333b2afaa1f103d874b449d: Status 404 returned error can't find the container with id aba843a1ed642a52ecf018eabf0d421bdd7d1ae11333b2afaa1f103d874b449d Jul 07 17:52:31.230440 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:31.230401 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" event={"ID":"61d2198e-3245-473d-9ee0-33e02270e9b0","Type":"ContainerStarted","Data":"dd73e6d8e0d19483ea6114adee188e3220f26a15aaedda3b224f719545a27533"} Jul 07 17:52:31.230440 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:31.230447 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" event={"ID":"61d2198e-3245-473d-9ee0-33e02270e9b0","Type":"ContainerStarted","Data":"aba843a1ed642a52ecf018eabf0d421bdd7d1ae11333b2afaa1f103d874b449d"} Jul 07 17:52:31.249871 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:31.249806 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" podStartSLOduration=1.972041279 podStartE2EDuration="2.249783712s" totalImagesPullingTime="277.742433ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:52:31.249204711 +0000 UTC m=+755.288531612" watchObservedRunningTime="2026-07-07 17:52:31.249783712 +0000 UTC m=+755.289110618" Jul 07 17:52:40.314985 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:40.314927 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:40.315417 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:40.315002 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:40.328112 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:40.328070 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:52:41.296251 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:52:41.296218 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l" Jul 07 17:53:56.386358 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:56.386321 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-67f644975-gc2wm"] Jul 07 17:53:56.389763 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:56.389742 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67f644975-gc2wm" Jul 07 17:53:56.393294 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:56.393264 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 07 17:53:56.393523 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:56.393367 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rzbhw\"" Jul 07 17:53:56.396637 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:56.396602 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67f644975-gc2wm"] Jul 07 17:53:56.430792 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:56.430757 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l24wc\" (UniqueName: \"kubernetes.io/projected/66e213cc-3700-4048-abba-f0ffc10c7116-kube-api-access-l24wc\") pod \"authorino-67f644975-gc2wm\" (UID: \"66e213cc-3700-4048-abba-f0ffc10c7116\") " pod="kuadrant-system/authorino-67f644975-gc2wm" Jul 07 17:53:56.430970 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:56.430808 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/66e213cc-3700-4048-abba-f0ffc10c7116-tls-cert\") pod \"authorino-67f644975-gc2wm\" (UID: \"66e213cc-3700-4048-abba-f0ffc10c7116\") " pod="kuadrant-system/authorino-67f644975-gc2wm" Jul 07 17:53:56.532259 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:56.532226 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/66e213cc-3700-4048-abba-f0ffc10c7116-tls-cert\") pod \"authorino-67f644975-gc2wm\" (UID: \"66e213cc-3700-4048-abba-f0ffc10c7116\") " pod="kuadrant-system/authorino-67f644975-gc2wm" Jul 07 17:53:56.532485 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:56.532451 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l24wc\" (UniqueName: \"kubernetes.io/projected/66e213cc-3700-4048-abba-f0ffc10c7116-kube-api-access-l24wc\") pod \"authorino-67f644975-gc2wm\" (UID: \"66e213cc-3700-4048-abba-f0ffc10c7116\") " pod="kuadrant-system/authorino-67f644975-gc2wm" Jul 07 17:53:56.540810 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:56.536013 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/66e213cc-3700-4048-abba-f0ffc10c7116-tls-cert\") pod \"authorino-67f644975-gc2wm\" (UID: \"66e213cc-3700-4048-abba-f0ffc10c7116\") " pod="kuadrant-system/authorino-67f644975-gc2wm" Jul 07 17:53:56.542301 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:56.542272 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l24wc\" (UniqueName: \"kubernetes.io/projected/66e213cc-3700-4048-abba-f0ffc10c7116-kube-api-access-l24wc\") pod \"authorino-67f644975-gc2wm\" (UID: \"66e213cc-3700-4048-abba-f0ffc10c7116\") " pod="kuadrant-system/authorino-67f644975-gc2wm" Jul 07 17:53:56.703138 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:56.703099 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67f644975-gc2wm" Jul 07 17:53:56.884550 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:53:56.884510 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66e213cc_3700_4048_abba_f0ffc10c7116.slice/crio-3640266c2591b322a3b6f90482639fd35a655995e73b772721b92d039acfe520 WatchSource:0}: Error finding container 3640266c2591b322a3b6f90482639fd35a655995e73b772721b92d039acfe520: Status 404 returned error can't find the container with id 3640266c2591b322a3b6f90482639fd35a655995e73b772721b92d039acfe520 Jul 07 17:53:56.888681 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:56.888652 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67f644975-gc2wm"] Jul 07 17:53:57.688398 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:57.688354 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67f644975-gc2wm" event={"ID":"66e213cc-3700-4048-abba-f0ffc10c7116","Type":"ContainerStarted","Data":"926f0d09aac31ca7aca01bab9cd923dc83f851992e8dda27ffcfe292d71a764a"} Jul 07 17:53:57.688814 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:57.688406 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67f644975-gc2wm" event={"ID":"66e213cc-3700-4048-abba-f0ffc10c7116","Type":"ContainerStarted","Data":"3640266c2591b322a3b6f90482639fd35a655995e73b772721b92d039acfe520"} Jul 07 17:53:57.713599 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:53:57.713534 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-67f644975-gc2wm" podStartSLOduration=1.351856682 podStartE2EDuration="1.713513899s" totalImagesPullingTime="361.657217ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:53:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:53:57.71180648 +0000 UTC m=+841.751133384" watchObservedRunningTime="2026-07-07 17:53:57.713513899 +0000 UTC m=+841.752840801" Jul 07 17:54:56.483273 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:54:56.483189 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 17:54:56.486195 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:54:56.486160 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 17:55:16.477144 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:16.477096 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76d699bd45-qvd76"] Jul 07 17:55:16.479690 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:16.477347 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-76d699bd45-qvd76" podUID="62c617c1-42c3-4804-a753-de89bc812f90" containerName="manager" containerID="cri-o://9dfbed3d7d2ec495390e9e64f51f5d0cb53770288b3bbd336759dda71d4fbb99" gracePeriod=30 Jul 07 17:55:16.746215 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:16.746134 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d699bd45-qvd76" Jul 07 17:55:16.855134 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:16.855097 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62c617c1-42c3-4804-a753-de89bc812f90-webhook-cert\") pod \"62c617c1-42c3-4804-a753-de89bc812f90\" (UID: \"62c617c1-42c3-4804-a753-de89bc812f90\") " Jul 07 17:55:16.855134 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:16.855141 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8fcx\" (UniqueName: \"kubernetes.io/projected/62c617c1-42c3-4804-a753-de89bc812f90-kube-api-access-q8fcx\") pod \"62c617c1-42c3-4804-a753-de89bc812f90\" (UID: \"62c617c1-42c3-4804-a753-de89bc812f90\") " Jul 07 17:55:16.857492 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:16.857448 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62c617c1-42c3-4804-a753-de89bc812f90-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "62c617c1-42c3-4804-a753-de89bc812f90" (UID: "62c617c1-42c3-4804-a753-de89bc812f90"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 17:55:16.857492 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:16.857484 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62c617c1-42c3-4804-a753-de89bc812f90-kube-api-access-q8fcx" (OuterVolumeSpecName: "kube-api-access-q8fcx") pod "62c617c1-42c3-4804-a753-de89bc812f90" (UID: "62c617c1-42c3-4804-a753-de89bc812f90"). InnerVolumeSpecName "kube-api-access-q8fcx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 17:55:16.956567 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:16.956531 2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62c617c1-42c3-4804-a753-de89bc812f90-webhook-cert\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:55:16.956567 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:16.956561 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8fcx\" (UniqueName: \"kubernetes.io/projected/62c617c1-42c3-4804-a753-de89bc812f90-kube-api-access-q8fcx\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 17:55:17.129416 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:17.129311 2577 generic.go:356] "Generic (PLEG): container finished" podID="62c617c1-42c3-4804-a753-de89bc812f90" containerID="9dfbed3d7d2ec495390e9e64f51f5d0cb53770288b3bbd336759dda71d4fbb99" exitCode=0 Jul 07 17:55:17.129416 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:17.129366 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d699bd45-qvd76" Jul 07 17:55:17.129658 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:17.129405 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d699bd45-qvd76" event={"ID":"62c617c1-42c3-4804-a753-de89bc812f90","Type":"ContainerDied","Data":"9dfbed3d7d2ec495390e9e64f51f5d0cb53770288b3bbd336759dda71d4fbb99"} Jul 07 17:55:17.129658 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:17.129479 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d699bd45-qvd76" event={"ID":"62c617c1-42c3-4804-a753-de89bc812f90","Type":"ContainerDied","Data":"c32b9a3cba609f71fc05c7584bdd75d7f0110801c59b36e110212a973d1ad2a9"} Jul 07 17:55:17.129658 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:17.129508 2577 scope.go:117] "RemoveContainer" containerID="9dfbed3d7d2ec495390e9e64f51f5d0cb53770288b3bbd336759dda71d4fbb99" Jul 07 17:55:17.139716 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:17.139695 2577 scope.go:117] "RemoveContainer" containerID="9dfbed3d7d2ec495390e9e64f51f5d0cb53770288b3bbd336759dda71d4fbb99" Jul 07 17:55:17.140055 ip-10-0-129-170 kubenswrapper[2577]: E0707 17:55:17.140022 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dfbed3d7d2ec495390e9e64f51f5d0cb53770288b3bbd336759dda71d4fbb99\": container with ID starting with 9dfbed3d7d2ec495390e9e64f51f5d0cb53770288b3bbd336759dda71d4fbb99 not found: ID does not exist" containerID="9dfbed3d7d2ec495390e9e64f51f5d0cb53770288b3bbd336759dda71d4fbb99" Jul 07 17:55:17.140127 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:17.140072 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dfbed3d7d2ec495390e9e64f51f5d0cb53770288b3bbd336759dda71d4fbb99"} err="failed to get container status \"9dfbed3d7d2ec495390e9e64f51f5d0cb53770288b3bbd336759dda71d4fbb99\": rpc error: code = NotFound desc = could not find container \"9dfbed3d7d2ec495390e9e64f51f5d0cb53770288b3bbd336759dda71d4fbb99\": container with ID starting with 9dfbed3d7d2ec495390e9e64f51f5d0cb53770288b3bbd336759dda71d4fbb99 not found: ID does not exist" Jul 07 17:55:17.151841 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:17.151800 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76d699bd45-qvd76"] Jul 07 17:55:17.156402 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:17.156361 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76d699bd45-qvd76"] Jul 07 17:55:18.048191 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.048145 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76d699bd45-zgtps"] Jul 07 17:55:18.048797 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.048696 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62c617c1-42c3-4804-a753-de89bc812f90" containerName="manager" Jul 07 17:55:18.048797 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.048717 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="62c617c1-42c3-4804-a753-de89bc812f90" containerName="manager" Jul 07 17:55:18.048797 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.048798 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="62c617c1-42c3-4804-a753-de89bc812f90" containerName="manager" Jul 07 17:55:18.053198 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.053166 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d699bd45-zgtps" Jul 07 17:55:18.056618 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.056588 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 17:55:18.056767 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.056588 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 17:55:18.056767 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.056592 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2xbz2\"" Jul 07 17:55:18.065214 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.065174 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76d699bd45-zgtps"] Jul 07 17:55:18.167456 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.167413 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b6ch\" (UniqueName: \"kubernetes.io/projected/89cadee5-d200-47b3-b0b8-05498c9fa8cf-kube-api-access-4b6ch\") pod \"maas-controller-76d699bd45-zgtps\" (UID: \"89cadee5-d200-47b3-b0b8-05498c9fa8cf\") " pod="opendatahub/maas-controller-76d699bd45-zgtps" Jul 07 17:55:18.167666 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.167547 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/89cadee5-d200-47b3-b0b8-05498c9fa8cf-webhook-cert\") pod \"maas-controller-76d699bd45-zgtps\" (UID: \"89cadee5-d200-47b3-b0b8-05498c9fa8cf\") " pod="opendatahub/maas-controller-76d699bd45-zgtps" Jul 07 17:55:18.268894 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.268850 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b6ch\" (UniqueName: \"kubernetes.io/projected/89cadee5-d200-47b3-b0b8-05498c9fa8cf-kube-api-access-4b6ch\") pod \"maas-controller-76d699bd45-zgtps\" (UID: \"89cadee5-d200-47b3-b0b8-05498c9fa8cf\") " pod="opendatahub/maas-controller-76d699bd45-zgtps" Jul 07 17:55:18.269117 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.268939 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/89cadee5-d200-47b3-b0b8-05498c9fa8cf-webhook-cert\") pod \"maas-controller-76d699bd45-zgtps\" (UID: \"89cadee5-d200-47b3-b0b8-05498c9fa8cf\") " pod="opendatahub/maas-controller-76d699bd45-zgtps" Jul 07 17:55:18.271659 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.271634 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/89cadee5-d200-47b3-b0b8-05498c9fa8cf-webhook-cert\") pod \"maas-controller-76d699bd45-zgtps\" (UID: \"89cadee5-d200-47b3-b0b8-05498c9fa8cf\") " pod="opendatahub/maas-controller-76d699bd45-zgtps" Jul 07 17:55:18.278296 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.278265 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b6ch\" (UniqueName: \"kubernetes.io/projected/89cadee5-d200-47b3-b0b8-05498c9fa8cf-kube-api-access-4b6ch\") pod \"maas-controller-76d699bd45-zgtps\" (UID: \"89cadee5-d200-47b3-b0b8-05498c9fa8cf\") " pod="opendatahub/maas-controller-76d699bd45-zgtps" Jul 07 17:55:18.366530 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.366421 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76d699bd45-zgtps" Jul 07 17:55:18.518302 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.518262 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62c617c1-42c3-4804-a753-de89bc812f90" path="/var/lib/kubelet/pods/62c617c1-42c3-4804-a753-de89bc812f90/volumes" Jul 07 17:55:18.518679 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:18.518659 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76d699bd45-zgtps"] Jul 07 17:55:18.519192 ip-10-0-129-170 kubenswrapper[2577]: W0707 17:55:18.519170 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89cadee5_d200_47b3_b0b8_05498c9fa8cf.slice/crio-c3d29984ee25c34084acf151eca4fcb3f313cb9a1c186ab4770d3b635a5d3c44 WatchSource:0}: Error finding container c3d29984ee25c34084acf151eca4fcb3f313cb9a1c186ab4770d3b635a5d3c44: Status 404 returned error can't find the container with id c3d29984ee25c34084acf151eca4fcb3f313cb9a1c186ab4770d3b635a5d3c44 Jul 07 17:55:19.143641 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:19.143598 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d699bd45-zgtps" event={"ID":"89cadee5-d200-47b3-b0b8-05498c9fa8cf","Type":"ContainerStarted","Data":"ea96cb6ad1f755f326467ed99b435c17f4e183d25b4134b05ea3add1244fedeb"} Jul 07 17:55:19.144066 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:19.143669 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76d699bd45-zgtps" event={"ID":"89cadee5-d200-47b3-b0b8-05498c9fa8cf","Type":"ContainerStarted","Data":"c3d29984ee25c34084acf151eca4fcb3f313cb9a1c186ab4770d3b635a5d3c44"} Jul 07 17:55:19.144066 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:19.143735 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76d699bd45-zgtps" Jul 07 17:55:19.161420 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:19.161359 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76d699bd45-zgtps" podStartSLOduration=0.6516987 podStartE2EDuration="1.161341525s" totalImagesPullingTime="509.642825ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:55:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:55:19.160120248 +0000 UTC m=+923.199447153" watchObservedRunningTime="2026-07-07 17:55:19.161341525 +0000 UTC m=+923.200668428" Jul 07 17:55:30.156382 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:55:30.156348 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76d699bd45-zgtps" Jul 07 17:59:56.529668 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:59:56.529639 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 17:59:56.530488 ip-10-0-129-170 kubenswrapper[2577]: I0707 17:59:56.530465 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 18:04:56.567612 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:04:56.567584 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 18:04:56.568250 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:04:56.567661 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 18:07:29.391854 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:29.391808 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r"] Jul 07 18:07:29.396108 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:29.396079 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:29.398864 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:29.398832 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-afd9b7ef-serving-cert\"" Jul 07 18:07:29.406245 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:29.406214 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r"] Jul 07 18:07:29.511796 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:29.511755 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/933674a4-9634-4bed-abd5-8d8b0c95fe57-openshift-service-ca\") pod \"maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r\" (UID: \"933674a4-9634-4bed-abd5-8d8b0c95fe57\") " pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:29.511796 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:29.511803 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/933674a4-9634-4bed-abd5-8d8b0c95fe57-maas-api-tls\") pod \"maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r\" (UID: \"933674a4-9634-4bed-abd5-8d8b0c95fe57\") " pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:29.512076 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:29.511980 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwgj5\" (UniqueName: \"kubernetes.io/projected/933674a4-9634-4bed-abd5-8d8b0c95fe57-kube-api-access-bwgj5\") pod \"maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r\" (UID: \"933674a4-9634-4bed-abd5-8d8b0c95fe57\") " pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:29.612554 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:29.612510 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/933674a4-9634-4bed-abd5-8d8b0c95fe57-maas-api-tls\") pod \"maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r\" (UID: \"933674a4-9634-4bed-abd5-8d8b0c95fe57\") " pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:29.612769 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:29.612634 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwgj5\" (UniqueName: \"kubernetes.io/projected/933674a4-9634-4bed-abd5-8d8b0c95fe57-kube-api-access-bwgj5\") pod \"maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r\" (UID: \"933674a4-9634-4bed-abd5-8d8b0c95fe57\") " pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:29.612769 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:29.612670 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/933674a4-9634-4bed-abd5-8d8b0c95fe57-openshift-service-ca\") pod \"maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r\" (UID: \"933674a4-9634-4bed-abd5-8d8b0c95fe57\") " pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:29.612769 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:07:29.612684 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-afd9b7ef-serving-cert: secret "maas-api-e2e-ait-afd9b7ef-serving-cert" not found Jul 07 18:07:29.612769 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:07:29.612766 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/933674a4-9634-4bed-abd5-8d8b0c95fe57-maas-api-tls podName:933674a4-9634-4bed-abd5-8d8b0c95fe57 nodeName:}" failed. No retries permitted until 2026-07-07 18:07:30.112742365 +0000 UTC m=+1654.152069265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/933674a4-9634-4bed-abd5-8d8b0c95fe57-maas-api-tls") pod "maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" (UID: "933674a4-9634-4bed-abd5-8d8b0c95fe57") : secret "maas-api-e2e-ait-afd9b7ef-serving-cert" not found Jul 07 18:07:29.613510 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:29.613482 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/933674a4-9634-4bed-abd5-8d8b0c95fe57-openshift-service-ca\") pod \"maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r\" (UID: \"933674a4-9634-4bed-abd5-8d8b0c95fe57\") " pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:29.621656 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:29.621624 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwgj5\" (UniqueName: \"kubernetes.io/projected/933674a4-9634-4bed-abd5-8d8b0c95fe57-kube-api-access-bwgj5\") pod \"maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r\" (UID: \"933674a4-9634-4bed-abd5-8d8b0c95fe57\") " pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:30.117370 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:30.117331 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/933674a4-9634-4bed-abd5-8d8b0c95fe57-maas-api-tls\") pod \"maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r\" (UID: \"933674a4-9634-4bed-abd5-8d8b0c95fe57\") " pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:30.120084 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:30.120054 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/933674a4-9634-4bed-abd5-8d8b0c95fe57-maas-api-tls\") pod \"maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r\" (UID: \"933674a4-9634-4bed-abd5-8d8b0c95fe57\") " pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:30.309588 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:30.309552 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:30.469319 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:30.469292 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r"] Jul 07 18:07:30.471750 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:07:30.471719 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod933674a4_9634_4bed_abd5_8d8b0c95fe57.slice/crio-0f58515e1fd464e24cd6932cd7149dba7b0a917ac4ca251d3793a52eb2541660 WatchSource:0}: Error finding container 0f58515e1fd464e24cd6932cd7149dba7b0a917ac4ca251d3793a52eb2541660: Status 404 returned error can't find the container with id 0f58515e1fd464e24cd6932cd7149dba7b0a917ac4ca251d3793a52eb2541660 Jul 07 18:07:30.473257 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:30.473235 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 18:07:31.229770 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:31.229659 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" event={"ID":"933674a4-9634-4bed-abd5-8d8b0c95fe57","Type":"ContainerStarted","Data":"8436ecd14f57636c140e6af90b32fe34e8f11995354a7c1be1e69ee8eb5ff8c0"} Jul 07 18:07:31.229770 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:31.229718 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:31.229770 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:31.229741 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" event={"ID":"933674a4-9634-4bed-abd5-8d8b0c95fe57","Type":"ContainerStarted","Data":"0f58515e1fd464e24cd6932cd7149dba7b0a917ac4ca251d3793a52eb2541660"} Jul 07 18:07:31.248831 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:31.248775 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" podStartSLOduration=1.752633495 podStartE2EDuration="2.248755046s" totalImagesPullingTime="496.121551ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:07:31.246560938 +0000 UTC m=+1655.285887854" watchObservedRunningTime="2026-07-07 18:07:31.248755046 +0000 UTC m=+1655.288081948" Jul 07 18:07:34.786246 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:34.786201 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r"] Jul 07 18:07:34.786722 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:34.786500 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" podUID="933674a4-9634-4bed-abd5-8d8b0c95fe57" containerName="maas-api" containerID="cri-o://8436ecd14f57636c140e6af90b32fe34e8f11995354a7c1be1e69ee8eb5ff8c0" gracePeriod=30 Jul 07 18:07:35.050017 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.049943 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:35.064387 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.064353 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/933674a4-9634-4bed-abd5-8d8b0c95fe57-maas-api-tls\") pod \"933674a4-9634-4bed-abd5-8d8b0c95fe57\" (UID: \"933674a4-9634-4bed-abd5-8d8b0c95fe57\") " Jul 07 18:07:35.064532 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.064407 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/933674a4-9634-4bed-abd5-8d8b0c95fe57-openshift-service-ca\") pod \"933674a4-9634-4bed-abd5-8d8b0c95fe57\" (UID: \"933674a4-9634-4bed-abd5-8d8b0c95fe57\") " Jul 07 18:07:35.064532 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.064426 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwgj5\" (UniqueName: \"kubernetes.io/projected/933674a4-9634-4bed-abd5-8d8b0c95fe57-kube-api-access-bwgj5\") pod \"933674a4-9634-4bed-abd5-8d8b0c95fe57\" (UID: \"933674a4-9634-4bed-abd5-8d8b0c95fe57\") " Jul 07 18:07:35.064761 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.064739 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/933674a4-9634-4bed-abd5-8d8b0c95fe57-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "933674a4-9634-4bed-abd5-8d8b0c95fe57" (UID: "933674a4-9634-4bed-abd5-8d8b0c95fe57"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:07:35.066982 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.066864 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/933674a4-9634-4bed-abd5-8d8b0c95fe57-kube-api-access-bwgj5" (OuterVolumeSpecName: "kube-api-access-bwgj5") pod "933674a4-9634-4bed-abd5-8d8b0c95fe57" (UID: "933674a4-9634-4bed-abd5-8d8b0c95fe57"). InnerVolumeSpecName "kube-api-access-bwgj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:07:35.066982 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.066944 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/933674a4-9634-4bed-abd5-8d8b0c95fe57-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "933674a4-9634-4bed-abd5-8d8b0c95fe57" (UID: "933674a4-9634-4bed-abd5-8d8b0c95fe57"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:07:35.165785 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.165747 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/933674a4-9634-4bed-abd5-8d8b0c95fe57-maas-api-tls\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:07:35.165785 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.165780 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/933674a4-9634-4bed-abd5-8d8b0c95fe57-openshift-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:07:35.165785 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.165791 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwgj5\" (UniqueName: \"kubernetes.io/projected/933674a4-9634-4bed-abd5-8d8b0c95fe57-kube-api-access-bwgj5\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:07:35.251612 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.251573 2577 generic.go:356] "Generic (PLEG): container finished" podID="933674a4-9634-4bed-abd5-8d8b0c95fe57" containerID="8436ecd14f57636c140e6af90b32fe34e8f11995354a7c1be1e69ee8eb5ff8c0" exitCode=0 Jul 07 18:07:35.251791 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.251623 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" Jul 07 18:07:35.251791 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.251669 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" event={"ID":"933674a4-9634-4bed-abd5-8d8b0c95fe57","Type":"ContainerDied","Data":"8436ecd14f57636c140e6af90b32fe34e8f11995354a7c1be1e69ee8eb5ff8c0"} Jul 07 18:07:35.251791 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.251724 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r" event={"ID":"933674a4-9634-4bed-abd5-8d8b0c95fe57","Type":"ContainerDied","Data":"0f58515e1fd464e24cd6932cd7149dba7b0a917ac4ca251d3793a52eb2541660"} Jul 07 18:07:35.251791 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.251748 2577 scope.go:117] "RemoveContainer" containerID="8436ecd14f57636c140e6af90b32fe34e8f11995354a7c1be1e69ee8eb5ff8c0" Jul 07 18:07:35.261978 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.261954 2577 scope.go:117] "RemoveContainer" containerID="8436ecd14f57636c140e6af90b32fe34e8f11995354a7c1be1e69ee8eb5ff8c0" Jul 07 18:07:35.262339 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:07:35.262321 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8436ecd14f57636c140e6af90b32fe34e8f11995354a7c1be1e69ee8eb5ff8c0\": container with ID starting with 8436ecd14f57636c140e6af90b32fe34e8f11995354a7c1be1e69ee8eb5ff8c0 not found: ID does not exist" containerID="8436ecd14f57636c140e6af90b32fe34e8f11995354a7c1be1e69ee8eb5ff8c0" Jul 07 18:07:35.262401 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.262355 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8436ecd14f57636c140e6af90b32fe34e8f11995354a7c1be1e69ee8eb5ff8c0"} err="failed to get container status \"8436ecd14f57636c140e6af90b32fe34e8f11995354a7c1be1e69ee8eb5ff8c0\": rpc error: code = NotFound desc = could not find container \"8436ecd14f57636c140e6af90b32fe34e8f11995354a7c1be1e69ee8eb5ff8c0\": container with ID starting with 8436ecd14f57636c140e6af90b32fe34e8f11995354a7c1be1e69ee8eb5ff8c0 not found: ID does not exist" Jul 07 18:07:35.274215 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.274177 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r"] Jul 07 18:07:35.281284 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:35.281250 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-afd9b7ef-57d97f64d5-dtx4r"] Jul 07 18:07:36.516925 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:36.516887 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="933674a4-9634-4bed-abd5-8d8b0c95fe57" path="/var/lib/kubelet/pods/933674a4-9634-4bed-abd5-8d8b0c95fe57/volumes" Jul 07 18:07:42.432499 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.432452 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9"] Jul 07 18:07:42.433077 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.433035 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="933674a4-9634-4bed-abd5-8d8b0c95fe57" containerName="maas-api" Jul 07 18:07:42.433133 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.433081 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="933674a4-9634-4bed-abd5-8d8b0c95fe57" containerName="maas-api" Jul 07 18:07:42.433246 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.433209 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="933674a4-9634-4bed-abd5-8d8b0c95fe57" containerName="maas-api" Jul 07 18:07:42.438225 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.438195 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" Jul 07 18:07:42.440753 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.440728 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-89337714-serving-cert\"" Jul 07 18:07:42.449176 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.449141 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9"] Jul 07 18:07:42.540420 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.540368 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cc770f-aba0-4ebd-ac1e-81be8709ef26-openshift-service-ca\") pod \"maas-api-e2e-ait-89337714-7c4446476c-hb6d9\" (UID: \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\") " pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" Jul 07 18:07:42.540420 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.540416 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftl2d\" (UniqueName: \"kubernetes.io/projected/29cc770f-aba0-4ebd-ac1e-81be8709ef26-kube-api-access-ftl2d\") pod \"maas-api-e2e-ait-89337714-7c4446476c-hb6d9\" (UID: \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\") " pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" Jul 07 18:07:42.540656 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.540441 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29cc770f-aba0-4ebd-ac1e-81be8709ef26-maas-api-tls\") pod \"maas-api-e2e-ait-89337714-7c4446476c-hb6d9\" (UID: \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\") " pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" Jul 07 18:07:42.641733 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.641678 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cc770f-aba0-4ebd-ac1e-81be8709ef26-openshift-service-ca\") pod \"maas-api-e2e-ait-89337714-7c4446476c-hb6d9\" (UID: \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\") " pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" Jul 07 18:07:42.641733 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.641731 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftl2d\" (UniqueName: \"kubernetes.io/projected/29cc770f-aba0-4ebd-ac1e-81be8709ef26-kube-api-access-ftl2d\") pod \"maas-api-e2e-ait-89337714-7c4446476c-hb6d9\" (UID: \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\") " pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" Jul 07 18:07:42.641984 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.641767 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29cc770f-aba0-4ebd-ac1e-81be8709ef26-maas-api-tls\") pod \"maas-api-e2e-ait-89337714-7c4446476c-hb6d9\" (UID: \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\") " pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" Jul 07 18:07:42.642591 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.642561 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cc770f-aba0-4ebd-ac1e-81be8709ef26-openshift-service-ca\") pod \"maas-api-e2e-ait-89337714-7c4446476c-hb6d9\" (UID: \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\") " pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" Jul 07 18:07:42.644549 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.644523 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29cc770f-aba0-4ebd-ac1e-81be8709ef26-maas-api-tls\") pod \"maas-api-e2e-ait-89337714-7c4446476c-hb6d9\" (UID: \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\") " pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" Jul 07 18:07:42.650772 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.650738 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftl2d\" (UniqueName: \"kubernetes.io/projected/29cc770f-aba0-4ebd-ac1e-81be8709ef26-kube-api-access-ftl2d\") pod \"maas-api-e2e-ait-89337714-7c4446476c-hb6d9\" (UID: \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\") " pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" Jul 07 18:07:42.752129 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.752007 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" Jul 07 18:07:42.913060 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:42.913005 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9"] Jul 07 18:07:42.914933 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:07:42.914893 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29cc770f_aba0_4ebd_ac1e_81be8709ef26.slice/crio-ccc9f442dca50c088426a62819655578cce6f5526b5f5a60331fd0c97a69c01d WatchSource:0}: Error finding container ccc9f442dca50c088426a62819655578cce6f5526b5f5a60331fd0c97a69c01d: Status 404 returned error can't find the container with id ccc9f442dca50c088426a62819655578cce6f5526b5f5a60331fd0c97a69c01d Jul 07 18:07:43.298311 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:43.298274 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" event={"ID":"29cc770f-aba0-4ebd-ac1e-81be8709ef26","Type":"ContainerStarted","Data":"ccc9f442dca50c088426a62819655578cce6f5526b5f5a60331fd0c97a69c01d"} Jul 07 18:07:44.305145 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:44.305105 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" event={"ID":"29cc770f-aba0-4ebd-ac1e-81be8709ef26","Type":"ContainerStarted","Data":"837ee2501e349fe62f4eef0a77f39a32cd91a715ee875667599fe4bb3e49135a"} Jul 07 18:07:44.305615 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:44.305168 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" Jul 07 18:07:44.326735 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:44.326676 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" podStartSLOduration=1.944783203 podStartE2EDuration="2.326656731s" totalImagesPullingTime="381.873528ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:07:44.325086484 +0000 UTC m=+1668.364413388" watchObservedRunningTime="2026-07-07 18:07:44.326656731 +0000 UTC m=+1668.365983634" Jul 07 18:07:47.863003 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:47.862955 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9"] Jul 07 18:07:47.863483 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:47.863182 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" podUID="29cc770f-aba0-4ebd-ac1e-81be8709ef26" containerName="maas-api" containerID="cri-o://837ee2501e349fe62f4eef0a77f39a32cd91a715ee875667599fe4bb3e49135a" gracePeriod=30 Jul 07 18:07:48.111643 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.111606 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" Jul 07 18:07:48.195248 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.195214 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cc770f-aba0-4ebd-ac1e-81be8709ef26-openshift-service-ca\") pod \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\" (UID: \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\") " Jul 07 18:07:48.195248 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.195255 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftl2d\" (UniqueName: \"kubernetes.io/projected/29cc770f-aba0-4ebd-ac1e-81be8709ef26-kube-api-access-ftl2d\") pod \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\" (UID: \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\") " Jul 07 18:07:48.195514 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.195391 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29cc770f-aba0-4ebd-ac1e-81be8709ef26-maas-api-tls\") pod \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\" (UID: \"29cc770f-aba0-4ebd-ac1e-81be8709ef26\") " Jul 07 18:07:48.195644 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.195613 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29cc770f-aba0-4ebd-ac1e-81be8709ef26-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "29cc770f-aba0-4ebd-ac1e-81be8709ef26" (UID: "29cc770f-aba0-4ebd-ac1e-81be8709ef26"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:07:48.197659 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.197612 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29cc770f-aba0-4ebd-ac1e-81be8709ef26-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "29cc770f-aba0-4ebd-ac1e-81be8709ef26" (UID: "29cc770f-aba0-4ebd-ac1e-81be8709ef26"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:07:48.197811 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.197763 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29cc770f-aba0-4ebd-ac1e-81be8709ef26-kube-api-access-ftl2d" (OuterVolumeSpecName: "kube-api-access-ftl2d") pod "29cc770f-aba0-4ebd-ac1e-81be8709ef26" (UID: "29cc770f-aba0-4ebd-ac1e-81be8709ef26"). InnerVolumeSpecName "kube-api-access-ftl2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:07:48.296448 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.296410 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29cc770f-aba0-4ebd-ac1e-81be8709ef26-maas-api-tls\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:07:48.296448 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.296441 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cc770f-aba0-4ebd-ac1e-81be8709ef26-openshift-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:07:48.296448 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.296452 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftl2d\" (UniqueName: \"kubernetes.io/projected/29cc770f-aba0-4ebd-ac1e-81be8709ef26-kube-api-access-ftl2d\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:07:48.329500 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.329461 2577 generic.go:356] "Generic (PLEG): container finished" podID="29cc770f-aba0-4ebd-ac1e-81be8709ef26" containerID="837ee2501e349fe62f4eef0a77f39a32cd91a715ee875667599fe4bb3e49135a" exitCode=0 Jul 07 18:07:48.329681 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.329507 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" Jul 07 18:07:48.329681 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.329550 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" event={"ID":"29cc770f-aba0-4ebd-ac1e-81be8709ef26","Type":"ContainerDied","Data":"837ee2501e349fe62f4eef0a77f39a32cd91a715ee875667599fe4bb3e49135a"} Jul 07 18:07:48.329681 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.329608 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9" event={"ID":"29cc770f-aba0-4ebd-ac1e-81be8709ef26","Type":"ContainerDied","Data":"ccc9f442dca50c088426a62819655578cce6f5526b5f5a60331fd0c97a69c01d"} Jul 07 18:07:48.329681 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.329635 2577 scope.go:117] "RemoveContainer" containerID="837ee2501e349fe62f4eef0a77f39a32cd91a715ee875667599fe4bb3e49135a" Jul 07 18:07:48.340340 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.340318 2577 scope.go:117] "RemoveContainer" containerID="837ee2501e349fe62f4eef0a77f39a32cd91a715ee875667599fe4bb3e49135a" Jul 07 18:07:48.340696 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:07:48.340674 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"837ee2501e349fe62f4eef0a77f39a32cd91a715ee875667599fe4bb3e49135a\": container with ID starting with 837ee2501e349fe62f4eef0a77f39a32cd91a715ee875667599fe4bb3e49135a not found: ID does not exist" containerID="837ee2501e349fe62f4eef0a77f39a32cd91a715ee875667599fe4bb3e49135a" Jul 07 18:07:48.340760 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.340712 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"837ee2501e349fe62f4eef0a77f39a32cd91a715ee875667599fe4bb3e49135a"} err="failed to get container status \"837ee2501e349fe62f4eef0a77f39a32cd91a715ee875667599fe4bb3e49135a\": rpc error: code = NotFound desc = could not find container \"837ee2501e349fe62f4eef0a77f39a32cd91a715ee875667599fe4bb3e49135a\": container with ID starting with 837ee2501e349fe62f4eef0a77f39a32cd91a715ee875667599fe4bb3e49135a not found: ID does not exist" Jul 07 18:07:48.352137 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.352098 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9"] Jul 07 18:07:48.355723 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.355688 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-89337714-7c4446476c-hb6d9"] Jul 07 18:07:48.516558 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:07:48.516460 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29cc770f-aba0-4ebd-ac1e-81be8709ef26" path="/var/lib/kubelet/pods/29cc770f-aba0-4ebd-ac1e-81be8709ef26/volumes" Jul 07 18:08:01.029869 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.029833 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk"] Jul 07 18:08:01.030329 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.030224 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29cc770f-aba0-4ebd-ac1e-81be8709ef26" containerName="maas-api" Jul 07 18:08:01.030329 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.030236 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="29cc770f-aba0-4ebd-ac1e-81be8709ef26" containerName="maas-api" Jul 07 18:08:01.030329 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.030298 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="29cc770f-aba0-4ebd-ac1e-81be8709ef26" containerName="maas-api" Jul 07 18:08:01.034782 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.034756 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:01.037324 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.037298 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-663072c9-serving-cert\"" Jul 07 18:08:01.044089 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.044060 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk"] Jul 07 18:08:01.113008 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.112966 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/463a990f-5c54-441c-bba2-9ad6f774f13f-openshift-service-ca\") pod \"maas-api-e2e-derive-663072c9-566d4d575d-2qbhk\" (UID: \"463a990f-5c54-441c-bba2-9ad6f774f13f\") " pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:01.113243 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.113067 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc74z\" (UniqueName: \"kubernetes.io/projected/463a990f-5c54-441c-bba2-9ad6f774f13f-kube-api-access-xc74z\") pod \"maas-api-e2e-derive-663072c9-566d4d575d-2qbhk\" (UID: \"463a990f-5c54-441c-bba2-9ad6f774f13f\") " pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:01.113243 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.113147 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/463a990f-5c54-441c-bba2-9ad6f774f13f-maas-api-tls\") pod \"maas-api-e2e-derive-663072c9-566d4d575d-2qbhk\" (UID: \"463a990f-5c54-441c-bba2-9ad6f774f13f\") " pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:01.214073 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.213993 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/463a990f-5c54-441c-bba2-9ad6f774f13f-openshift-service-ca\") pod \"maas-api-e2e-derive-663072c9-566d4d575d-2qbhk\" (UID: \"463a990f-5c54-441c-bba2-9ad6f774f13f\") " pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:01.214278 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.214126 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc74z\" (UniqueName: \"kubernetes.io/projected/463a990f-5c54-441c-bba2-9ad6f774f13f-kube-api-access-xc74z\") pod \"maas-api-e2e-derive-663072c9-566d4d575d-2qbhk\" (UID: \"463a990f-5c54-441c-bba2-9ad6f774f13f\") " pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:01.214278 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.214189 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/463a990f-5c54-441c-bba2-9ad6f774f13f-maas-api-tls\") pod \"maas-api-e2e-derive-663072c9-566d4d575d-2qbhk\" (UID: \"463a990f-5c54-441c-bba2-9ad6f774f13f\") " pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:01.214409 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:08:01.214342 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-663072c9-serving-cert: secret "maas-api-e2e-derive-663072c9-serving-cert" not found Jul 07 18:08:01.214462 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:08:01.214411 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/463a990f-5c54-441c-bba2-9ad6f774f13f-maas-api-tls podName:463a990f-5c54-441c-bba2-9ad6f774f13f nodeName:}" failed. No retries permitted until 2026-07-07 18:08:01.714387703 +0000 UTC m=+1685.753714599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/463a990f-5c54-441c-bba2-9ad6f774f13f-maas-api-tls") pod "maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" (UID: "463a990f-5c54-441c-bba2-9ad6f774f13f") : secret "maas-api-e2e-derive-663072c9-serving-cert" not found Jul 07 18:08:01.214672 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.214646 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/463a990f-5c54-441c-bba2-9ad6f774f13f-openshift-service-ca\") pod \"maas-api-e2e-derive-663072c9-566d4d575d-2qbhk\" (UID: \"463a990f-5c54-441c-bba2-9ad6f774f13f\") " pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:01.224621 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.224573 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc74z\" (UniqueName: \"kubernetes.io/projected/463a990f-5c54-441c-bba2-9ad6f774f13f-kube-api-access-xc74z\") pod \"maas-api-e2e-derive-663072c9-566d4d575d-2qbhk\" (UID: \"463a990f-5c54-441c-bba2-9ad6f774f13f\") " pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:01.718295 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.718253 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/463a990f-5c54-441c-bba2-9ad6f774f13f-maas-api-tls\") pod \"maas-api-e2e-derive-663072c9-566d4d575d-2qbhk\" (UID: \"463a990f-5c54-441c-bba2-9ad6f774f13f\") " pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:01.721119 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.721087 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/463a990f-5c54-441c-bba2-9ad6f774f13f-maas-api-tls\") pod \"maas-api-e2e-derive-663072c9-566d4d575d-2qbhk\" (UID: \"463a990f-5c54-441c-bba2-9ad6f774f13f\") " pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:01.947852 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:01.947808 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:02.103462 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:02.103434 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk"] Jul 07 18:08:02.106524 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:08:02.106493 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod463a990f_5c54_441c_bba2_9ad6f774f13f.slice/crio-75eb3b6c2174a1cc7be1da0f9ad77de4fae50ce78ca1e1d3e011540801aa34b1 WatchSource:0}: Error finding container 75eb3b6c2174a1cc7be1da0f9ad77de4fae50ce78ca1e1d3e011540801aa34b1: Status 404 returned error can't find the container with id 75eb3b6c2174a1cc7be1da0f9ad77de4fae50ce78ca1e1d3e011540801aa34b1 Jul 07 18:08:02.413440 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:02.413343 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" event={"ID":"463a990f-5c54-441c-bba2-9ad6f774f13f","Type":"ContainerStarted","Data":"75eb3b6c2174a1cc7be1da0f9ad77de4fae50ce78ca1e1d3e011540801aa34b1"} Jul 07 18:08:03.421215 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:03.421180 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" event={"ID":"463a990f-5c54-441c-bba2-9ad6f774f13f","Type":"ContainerStarted","Data":"9e52a14c97bac5542fc1d5b45c8720279ae8700e974345dc17c09727c872baa2"} Jul 07 18:08:03.421658 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:03.421292 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:03.440455 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:03.440407 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" podStartSLOduration=2.07112585 podStartE2EDuration="2.440390979s" totalImagesPullingTime="369.265129ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:08:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:08:03.439499618 +0000 UTC m=+1687.478826523" watchObservedRunningTime="2026-07-07 18:08:03.440390979 +0000 UTC m=+1687.479717889" Jul 07 18:08:05.815054 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:05.815008 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk"] Jul 07 18:08:05.815539 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:05.815296 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" podUID="463a990f-5c54-441c-bba2-9ad6f774f13f" containerName="maas-api" containerID="cri-o://9e52a14c97bac5542fc1d5b45c8720279ae8700e974345dc17c09727c872baa2" gracePeriod=30 Jul 07 18:08:06.073626 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.073540 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:06.162882 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.162843 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/463a990f-5c54-441c-bba2-9ad6f774f13f-maas-api-tls\") pod \"463a990f-5c54-441c-bba2-9ad6f774f13f\" (UID: \"463a990f-5c54-441c-bba2-9ad6f774f13f\") " Jul 07 18:08:06.163090 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.162988 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xc74z\" (UniqueName: \"kubernetes.io/projected/463a990f-5c54-441c-bba2-9ad6f774f13f-kube-api-access-xc74z\") pod \"463a990f-5c54-441c-bba2-9ad6f774f13f\" (UID: \"463a990f-5c54-441c-bba2-9ad6f774f13f\") " Jul 07 18:08:06.163090 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.163010 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/463a990f-5c54-441c-bba2-9ad6f774f13f-openshift-service-ca\") pod \"463a990f-5c54-441c-bba2-9ad6f774f13f\" (UID: \"463a990f-5c54-441c-bba2-9ad6f774f13f\") " Jul 07 18:08:06.163458 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.163436 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/463a990f-5c54-441c-bba2-9ad6f774f13f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "463a990f-5c54-441c-bba2-9ad6f774f13f" (UID: "463a990f-5c54-441c-bba2-9ad6f774f13f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:08:06.165359 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.165330 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/463a990f-5c54-441c-bba2-9ad6f774f13f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "463a990f-5c54-441c-bba2-9ad6f774f13f" (UID: "463a990f-5c54-441c-bba2-9ad6f774f13f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:08:06.165456 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.165406 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/463a990f-5c54-441c-bba2-9ad6f774f13f-kube-api-access-xc74z" (OuterVolumeSpecName: "kube-api-access-xc74z") pod "463a990f-5c54-441c-bba2-9ad6f774f13f" (UID: "463a990f-5c54-441c-bba2-9ad6f774f13f"). InnerVolumeSpecName "kube-api-access-xc74z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:08:06.264102 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.264024 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xc74z\" (UniqueName: \"kubernetes.io/projected/463a990f-5c54-441c-bba2-9ad6f774f13f-kube-api-access-xc74z\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:08:06.264102 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.264094 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/463a990f-5c54-441c-bba2-9ad6f774f13f-openshift-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:08:06.264102 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.264107 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/463a990f-5c54-441c-bba2-9ad6f774f13f-maas-api-tls\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:08:06.440977 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.440943 2577 generic.go:356] "Generic (PLEG): container finished" podID="463a990f-5c54-441c-bba2-9ad6f774f13f" containerID="9e52a14c97bac5542fc1d5b45c8720279ae8700e974345dc17c09727c872baa2" exitCode=0 Jul 07 18:08:06.441226 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.440997 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" Jul 07 18:08:06.441226 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.441025 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" event={"ID":"463a990f-5c54-441c-bba2-9ad6f774f13f","Type":"ContainerDied","Data":"9e52a14c97bac5542fc1d5b45c8720279ae8700e974345dc17c09727c872baa2"} Jul 07 18:08:06.441226 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.441108 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk" event={"ID":"463a990f-5c54-441c-bba2-9ad6f774f13f","Type":"ContainerDied","Data":"75eb3b6c2174a1cc7be1da0f9ad77de4fae50ce78ca1e1d3e011540801aa34b1"} Jul 07 18:08:06.441226 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.441133 2577 scope.go:117] "RemoveContainer" containerID="9e52a14c97bac5542fc1d5b45c8720279ae8700e974345dc17c09727c872baa2" Jul 07 18:08:06.451601 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.451579 2577 scope.go:117] "RemoveContainer" containerID="9e52a14c97bac5542fc1d5b45c8720279ae8700e974345dc17c09727c872baa2" Jul 07 18:08:06.451941 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:08:06.451918 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e52a14c97bac5542fc1d5b45c8720279ae8700e974345dc17c09727c872baa2\": container with ID starting with 9e52a14c97bac5542fc1d5b45c8720279ae8700e974345dc17c09727c872baa2 not found: ID does not exist" containerID="9e52a14c97bac5542fc1d5b45c8720279ae8700e974345dc17c09727c872baa2" Jul 07 18:08:06.452008 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.451952 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e52a14c97bac5542fc1d5b45c8720279ae8700e974345dc17c09727c872baa2"} err="failed to get container status \"9e52a14c97bac5542fc1d5b45c8720279ae8700e974345dc17c09727c872baa2\": rpc error: code = NotFound desc = could not find container \"9e52a14c97bac5542fc1d5b45c8720279ae8700e974345dc17c09727c872baa2\": container with ID starting with 9e52a14c97bac5542fc1d5b45c8720279ae8700e974345dc17c09727c872baa2 not found: ID does not exist" Jul 07 18:08:06.463874 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.463832 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk"] Jul 07 18:08:06.468684 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.468644 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-663072c9-566d4d575d-2qbhk"] Jul 07 18:08:06.518737 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:08:06.518696 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="463a990f-5c54-441c-bba2-9ad6f774f13f" path="/var/lib/kubelet/pods/463a990f-5c54-441c-bba2-9ad6f774f13f/volumes" Jul 07 18:09:52.205831 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.205739 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh"] Jul 07 18:09:52.206367 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.206254 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="463a990f-5c54-441c-bba2-9ad6f774f13f" containerName="maas-api" Jul 07 18:09:52.206367 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.206274 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="463a990f-5c54-441c-bba2-9ad6f774f13f" containerName="maas-api" Jul 07 18:09:52.206482 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.206386 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="463a990f-5c54-441c-bba2-9ad6f774f13f" containerName="maas-api" Jul 07 18:09:52.209722 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.209700 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:09:52.214389 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.214363 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-5f113509-serving-cert\"" Jul 07 18:09:52.230777 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.230740 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh"] Jul 07 18:09:52.244245 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.244212 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-openshift-service-ca\") pod \"maas-api-e2e-mt-5f113509-69497d59c6-4frxh\" (UID: \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\") " pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:09:52.244443 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.244275 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n89wg\" (UniqueName: \"kubernetes.io/projected/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-kube-api-access-n89wg\") pod \"maas-api-e2e-mt-5f113509-69497d59c6-4frxh\" (UID: \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\") " pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:09:52.244443 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.244346 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-maas-api-tls\") pod \"maas-api-e2e-mt-5f113509-69497d59c6-4frxh\" (UID: \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\") " pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:09:52.345467 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.345421 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n89wg\" (UniqueName: \"kubernetes.io/projected/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-kube-api-access-n89wg\") pod \"maas-api-e2e-mt-5f113509-69497d59c6-4frxh\" (UID: \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\") " pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:09:52.345687 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.345501 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-maas-api-tls\") pod \"maas-api-e2e-mt-5f113509-69497d59c6-4frxh\" (UID: \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\") " pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:09:52.345687 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.345580 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-openshift-service-ca\") pod \"maas-api-e2e-mt-5f113509-69497d59c6-4frxh\" (UID: \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\") " pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:09:52.345687 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:09:52.345644 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-5f113509-serving-cert: secret "maas-api-e2e-mt-5f113509-serving-cert" not found Jul 07 18:09:52.345844 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:09:52.345720 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-maas-api-tls podName:c7a280df-d9c4-4ba7-9005-dfac0d48e8dd nodeName:}" failed. No retries permitted until 2026-07-07 18:09:52.84570334 +0000 UTC m=+1796.885030221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-maas-api-tls") pod "maas-api-e2e-mt-5f113509-69497d59c6-4frxh" (UID: "c7a280df-d9c4-4ba7-9005-dfac0d48e8dd") : secret "maas-api-e2e-mt-5f113509-serving-cert" not found Jul 07 18:09:52.346305 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.346287 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-openshift-service-ca\") pod \"maas-api-e2e-mt-5f113509-69497d59c6-4frxh\" (UID: \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\") " pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:09:52.360816 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.360775 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n89wg\" (UniqueName: \"kubernetes.io/projected/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-kube-api-access-n89wg\") pod \"maas-api-e2e-mt-5f113509-69497d59c6-4frxh\" (UID: \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\") " pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:09:52.852334 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.852278 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-maas-api-tls\") pod \"maas-api-e2e-mt-5f113509-69497d59c6-4frxh\" (UID: \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\") " pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:09:52.855001 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:52.854961 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-maas-api-tls\") pod \"maas-api-e2e-mt-5f113509-69497d59c6-4frxh\" (UID: \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\") " pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:09:53.120582 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:53.120484 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:09:53.276204 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:53.276168 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh"] Jul 07 18:09:53.280295 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:09:53.280266 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7a280df_d9c4_4ba7_9005_dfac0d48e8dd.slice/crio-f519ad4d7e127af9175ae1a5657dab63b49fabf3a6236daa398daf3b9d45cc01 WatchSource:0}: Error finding container f519ad4d7e127af9175ae1a5657dab63b49fabf3a6236daa398daf3b9d45cc01: Status 404 returned error can't find the container with id f519ad4d7e127af9175ae1a5657dab63b49fabf3a6236daa398daf3b9d45cc01 Jul 07 18:09:54.031924 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:54.031884 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" event={"ID":"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd","Type":"ContainerStarted","Data":"a4648bb297533c8fcb0d2cd2ab380affb0686830fdf593aedc374e46f97c72ee"} Jul 07 18:09:54.032163 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:54.031947 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" event={"ID":"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd","Type":"ContainerStarted","Data":"f519ad4d7e127af9175ae1a5657dab63b49fabf3a6236daa398daf3b9d45cc01"} Jul 07 18:09:54.032163 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:54.031978 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:09:54.052340 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:54.052282 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" podStartSLOduration=1.3891414420000001 podStartE2EDuration="2.05226347s" totalImagesPullingTime="663.122028ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:09:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:09:54.049400812 +0000 UTC m=+1798.088727706" watchObservedRunningTime="2026-07-07 18:09:54.05226347 +0000 UTC m=+1798.091590373" Jul 07 18:09:56.601952 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:56.601924 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 18:09:56.603599 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:09:56.603576 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 18:10:00.043826 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:00.043794 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:10:04.154995 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:04.154948 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh"] Jul 07 18:10:04.155472 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:04.155268 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" podUID="c7a280df-d9c4-4ba7-9005-dfac0d48e8dd" containerName="maas-api" containerID="cri-o://a4648bb297533c8fcb0d2cd2ab380affb0686830fdf593aedc374e46f97c72ee" gracePeriod=30 Jul 07 18:10:04.432626 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:04.432596 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:10:04.559125 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:04.559077 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n89wg\" (UniqueName: \"kubernetes.io/projected/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-kube-api-access-n89wg\") pod \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\" (UID: \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\") " Jul 07 18:10:04.559345 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:04.559195 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-maas-api-tls\") pod \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\" (UID: \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\") " Jul 07 18:10:04.559345 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:04.559248 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-openshift-service-ca\") pod \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\" (UID: \"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd\") " Jul 07 18:10:04.559671 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:04.559643 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c7a280df-d9c4-4ba7-9005-dfac0d48e8dd" (UID: "c7a280df-d9c4-4ba7-9005-dfac0d48e8dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:10:04.561797 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:04.561749 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-kube-api-access-n89wg" (OuterVolumeSpecName: "kube-api-access-n89wg") pod "c7a280df-d9c4-4ba7-9005-dfac0d48e8dd" (UID: "c7a280df-d9c4-4ba7-9005-dfac0d48e8dd"). InnerVolumeSpecName "kube-api-access-n89wg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:10:04.561943 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:04.561810 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c7a280df-d9c4-4ba7-9005-dfac0d48e8dd" (UID: "c7a280df-d9c4-4ba7-9005-dfac0d48e8dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:10:04.660311 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:04.660201 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n89wg\" (UniqueName: \"kubernetes.io/projected/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-kube-api-access-n89wg\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:10:04.660311 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:04.660238 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-maas-api-tls\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:10:04.660311 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:04.660253 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd-openshift-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:10:05.095006 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:05.094972 2577 generic.go:356] "Generic (PLEG): container finished" podID="c7a280df-d9c4-4ba7-9005-dfac0d48e8dd" containerID="a4648bb297533c8fcb0d2cd2ab380affb0686830fdf593aedc374e46f97c72ee" exitCode=0 Jul 07 18:10:05.095226 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:05.095023 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" Jul 07 18:10:05.095226 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:05.095067 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" event={"ID":"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd","Type":"ContainerDied","Data":"a4648bb297533c8fcb0d2cd2ab380affb0686830fdf593aedc374e46f97c72ee"} Jul 07 18:10:05.095226 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:05.095130 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh" event={"ID":"c7a280df-d9c4-4ba7-9005-dfac0d48e8dd","Type":"ContainerDied","Data":"f519ad4d7e127af9175ae1a5657dab63b49fabf3a6236daa398daf3b9d45cc01"} Jul 07 18:10:05.095226 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:05.095158 2577 scope.go:117] "RemoveContainer" containerID="a4648bb297533c8fcb0d2cd2ab380affb0686830fdf593aedc374e46f97c72ee" Jul 07 18:10:05.105265 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:05.105244 2577 scope.go:117] "RemoveContainer" containerID="a4648bb297533c8fcb0d2cd2ab380affb0686830fdf593aedc374e46f97c72ee" Jul 07 18:10:05.105569 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:10:05.105551 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4648bb297533c8fcb0d2cd2ab380affb0686830fdf593aedc374e46f97c72ee\": container with ID starting with a4648bb297533c8fcb0d2cd2ab380affb0686830fdf593aedc374e46f97c72ee not found: ID does not exist" containerID="a4648bb297533c8fcb0d2cd2ab380affb0686830fdf593aedc374e46f97c72ee" Jul 07 18:10:05.105626 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:05.105577 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4648bb297533c8fcb0d2cd2ab380affb0686830fdf593aedc374e46f97c72ee"} err="failed to get container status \"a4648bb297533c8fcb0d2cd2ab380affb0686830fdf593aedc374e46f97c72ee\": rpc error: code = NotFound desc = could not find container \"a4648bb297533c8fcb0d2cd2ab380affb0686830fdf593aedc374e46f97c72ee\": container with ID starting with a4648bb297533c8fcb0d2cd2ab380affb0686830fdf593aedc374e46f97c72ee not found: ID does not exist" Jul 07 18:10:05.117914 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:05.117878 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh"] Jul 07 18:10:05.122899 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:05.122859 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-5f113509-69497d59c6-4frxh"] Jul 07 18:10:06.516670 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:06.516626 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7a280df-d9c4-4ba7-9005-dfac0d48e8dd" path="/var/lib/kubelet/pods/c7a280df-d9c4-4ba7-9005-dfac0d48e8dd/volumes" Jul 07 18:10:33.127720 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.127672 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl"] Jul 07 18:10:33.128253 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.128217 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7a280df-d9c4-4ba7-9005-dfac0d48e8dd" containerName="maas-api" Jul 07 18:10:33.128253 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.128233 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7a280df-d9c4-4ba7-9005-dfac0d48e8dd" containerName="maas-api" Jul 07 18:10:33.128348 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.128309 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7a280df-d9c4-4ba7-9005-dfac0d48e8dd" containerName="maas-api" Jul 07 18:10:33.131560 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.131538 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:10:33.134530 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.134505 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-8aa60534-serving-cert\"" Jul 07 18:10:33.144077 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.144018 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl"] Jul 07 18:10:33.319692 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.319647 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29abf51f-23ae-4337-9cc4-7275f7aacd44-maas-api-tls\") pod \"maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl\" (UID: \"29abf51f-23ae-4337-9cc4-7275f7aacd44\") " pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:10:33.319906 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.319769 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52x7p\" (UniqueName: \"kubernetes.io/projected/29abf51f-23ae-4337-9cc4-7275f7aacd44-kube-api-access-52x7p\") pod \"maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl\" (UID: \"29abf51f-23ae-4337-9cc4-7275f7aacd44\") " pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:10:33.319906 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.319814 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29abf51f-23ae-4337-9cc4-7275f7aacd44-openshift-service-ca\") pod \"maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl\" (UID: \"29abf51f-23ae-4337-9cc4-7275f7aacd44\") " pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:10:33.421271 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.421163 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52x7p\" (UniqueName: \"kubernetes.io/projected/29abf51f-23ae-4337-9cc4-7275f7aacd44-kube-api-access-52x7p\") pod \"maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl\" (UID: \"29abf51f-23ae-4337-9cc4-7275f7aacd44\") " pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:10:33.421271 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.421223 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29abf51f-23ae-4337-9cc4-7275f7aacd44-openshift-service-ca\") pod \"maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl\" (UID: \"29abf51f-23ae-4337-9cc4-7275f7aacd44\") " pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:10:33.421500 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.421309 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29abf51f-23ae-4337-9cc4-7275f7aacd44-maas-api-tls\") pod \"maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl\" (UID: \"29abf51f-23ae-4337-9cc4-7275f7aacd44\") " pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:10:33.421500 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:10:33.421489 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-8aa60534-serving-cert: secret "maas-api-e2e-mt-8aa60534-serving-cert" not found Jul 07 18:10:33.421604 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:10:33.421569 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29abf51f-23ae-4337-9cc4-7275f7aacd44-maas-api-tls podName:29abf51f-23ae-4337-9cc4-7275f7aacd44 nodeName:}" failed. No retries permitted until 2026-07-07 18:10:33.921552423 +0000 UTC m=+1837.960879304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/29abf51f-23ae-4337-9cc4-7275f7aacd44-maas-api-tls") pod "maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" (UID: "29abf51f-23ae-4337-9cc4-7275f7aacd44") : secret "maas-api-e2e-mt-8aa60534-serving-cert" not found Jul 07 18:10:33.421993 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.421971 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29abf51f-23ae-4337-9cc4-7275f7aacd44-openshift-service-ca\") pod \"maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl\" (UID: \"29abf51f-23ae-4337-9cc4-7275f7aacd44\") " pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:10:33.433158 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.433126 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52x7p\" (UniqueName: \"kubernetes.io/projected/29abf51f-23ae-4337-9cc4-7275f7aacd44-kube-api-access-52x7p\") pod \"maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl\" (UID: \"29abf51f-23ae-4337-9cc4-7275f7aacd44\") " pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:10:33.925321 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.925278 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29abf51f-23ae-4337-9cc4-7275f7aacd44-maas-api-tls\") pod \"maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl\" (UID: \"29abf51f-23ae-4337-9cc4-7275f7aacd44\") " pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:10:33.928123 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:33.928078 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29abf51f-23ae-4337-9cc4-7275f7aacd44-maas-api-tls\") pod \"maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl\" (UID: \"29abf51f-23ae-4337-9cc4-7275f7aacd44\") " pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:10:34.043132 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:34.043095 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:10:34.199203 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:34.199167 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl"] Jul 07 18:10:34.203054 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:10:34.202998 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29abf51f_23ae_4337_9cc4_7275f7aacd44.slice/crio-c3968be1e053036e8a83f9dd1b612f4b596ad497218cee01b9a106d111cd0c17 WatchSource:0}: Error finding container c3968be1e053036e8a83f9dd1b612f4b596ad497218cee01b9a106d111cd0c17: Status 404 returned error can't find the container with id c3968be1e053036e8a83f9dd1b612f4b596ad497218cee01b9a106d111cd0c17 Jul 07 18:10:34.264914 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:34.264867 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" event={"ID":"29abf51f-23ae-4337-9cc4-7275f7aacd44","Type":"ContainerStarted","Data":"c3968be1e053036e8a83f9dd1b612f4b596ad497218cee01b9a106d111cd0c17"} Jul 07 18:10:35.280727 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:35.280679 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" event={"ID":"29abf51f-23ae-4337-9cc4-7275f7aacd44","Type":"ContainerStarted","Data":"61edd64fceaa765d3a381f0026d1da7f79aae02add5f51982522ce4cab229db7"} Jul 07 18:10:35.281172 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:35.280807 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:10:35.300361 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:35.300297 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" podStartSLOduration=1.854788823 podStartE2EDuration="2.300278363s" totalImagesPullingTime="445.48954ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:10:35.298113323 +0000 UTC m=+1839.337440227" watchObservedRunningTime="2026-07-07 18:10:35.300278363 +0000 UTC m=+1839.339605266" Jul 07 18:10:41.293274 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:10:41.293236 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:11:25.918830 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:25.918734 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl"] Jul 07 18:11:25.919419 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:25.919018 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" podUID="29abf51f-23ae-4337-9cc4-7275f7aacd44" containerName="maas-api" containerID="cri-o://61edd64fceaa765d3a381f0026d1da7f79aae02add5f51982522ce4cab229db7" gracePeriod=30 Jul 07 18:11:26.177527 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.177447 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:11:26.218984 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.218937 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29abf51f-23ae-4337-9cc4-7275f7aacd44-maas-api-tls\") pod \"29abf51f-23ae-4337-9cc4-7275f7aacd44\" (UID: \"29abf51f-23ae-4337-9cc4-7275f7aacd44\") " Jul 07 18:11:26.219193 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.219089 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52x7p\" (UniqueName: \"kubernetes.io/projected/29abf51f-23ae-4337-9cc4-7275f7aacd44-kube-api-access-52x7p\") pod \"29abf51f-23ae-4337-9cc4-7275f7aacd44\" (UID: \"29abf51f-23ae-4337-9cc4-7275f7aacd44\") " Jul 07 18:11:26.219193 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.219141 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29abf51f-23ae-4337-9cc4-7275f7aacd44-openshift-service-ca\") pod \"29abf51f-23ae-4337-9cc4-7275f7aacd44\" (UID: \"29abf51f-23ae-4337-9cc4-7275f7aacd44\") " Jul 07 18:11:26.219530 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.219506 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29abf51f-23ae-4337-9cc4-7275f7aacd44-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "29abf51f-23ae-4337-9cc4-7275f7aacd44" (UID: "29abf51f-23ae-4337-9cc4-7275f7aacd44"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:11:26.221465 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.221440 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29abf51f-23ae-4337-9cc4-7275f7aacd44-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "29abf51f-23ae-4337-9cc4-7275f7aacd44" (UID: "29abf51f-23ae-4337-9cc4-7275f7aacd44"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:11:26.221574 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.221485 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29abf51f-23ae-4337-9cc4-7275f7aacd44-kube-api-access-52x7p" (OuterVolumeSpecName: "kube-api-access-52x7p") pod "29abf51f-23ae-4337-9cc4-7275f7aacd44" (UID: "29abf51f-23ae-4337-9cc4-7275f7aacd44"). InnerVolumeSpecName "kube-api-access-52x7p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:11:26.320399 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.320363 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52x7p\" (UniqueName: \"kubernetes.io/projected/29abf51f-23ae-4337-9cc4-7275f7aacd44-kube-api-access-52x7p\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:11:26.320399 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.320394 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29abf51f-23ae-4337-9cc4-7275f7aacd44-openshift-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:11:26.320399 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.320405 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29abf51f-23ae-4337-9cc4-7275f7aacd44-maas-api-tls\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:11:26.567579 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.567540 2577 generic.go:356] "Generic (PLEG): container finished" podID="29abf51f-23ae-4337-9cc4-7275f7aacd44" containerID="61edd64fceaa765d3a381f0026d1da7f79aae02add5f51982522ce4cab229db7" exitCode=0 Jul 07 18:11:26.567755 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.567588 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" Jul 07 18:11:26.567755 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.567616 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" event={"ID":"29abf51f-23ae-4337-9cc4-7275f7aacd44","Type":"ContainerDied","Data":"61edd64fceaa765d3a381f0026d1da7f79aae02add5f51982522ce4cab229db7"} Jul 07 18:11:26.567755 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.567677 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl" event={"ID":"29abf51f-23ae-4337-9cc4-7275f7aacd44","Type":"ContainerDied","Data":"c3968be1e053036e8a83f9dd1b612f4b596ad497218cee01b9a106d111cd0c17"} Jul 07 18:11:26.567755 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.567700 2577 scope.go:117] "RemoveContainer" containerID="61edd64fceaa765d3a381f0026d1da7f79aae02add5f51982522ce4cab229db7" Jul 07 18:11:26.577581 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.577557 2577 scope.go:117] "RemoveContainer" containerID="61edd64fceaa765d3a381f0026d1da7f79aae02add5f51982522ce4cab229db7" Jul 07 18:11:26.577894 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:11:26.577875 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61edd64fceaa765d3a381f0026d1da7f79aae02add5f51982522ce4cab229db7\": container with ID starting with 61edd64fceaa765d3a381f0026d1da7f79aae02add5f51982522ce4cab229db7 not found: ID does not exist" containerID="61edd64fceaa765d3a381f0026d1da7f79aae02add5f51982522ce4cab229db7" Jul 07 18:11:26.577950 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.577916 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61edd64fceaa765d3a381f0026d1da7f79aae02add5f51982522ce4cab229db7"} err="failed to get container status \"61edd64fceaa765d3a381f0026d1da7f79aae02add5f51982522ce4cab229db7\": rpc error: code = NotFound desc = could not find container \"61edd64fceaa765d3a381f0026d1da7f79aae02add5f51982522ce4cab229db7\": container with ID starting with 61edd64fceaa765d3a381f0026d1da7f79aae02add5f51982522ce4cab229db7 not found: ID does not exist" Jul 07 18:11:26.586754 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.586718 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl"] Jul 07 18:11:26.591645 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:26.591602 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-8aa60534-774f9ccf8b-prsfl"] Jul 07 18:11:28.519909 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:11:28.519863 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29abf51f-23ae-4337-9cc4-7275f7aacd44" path="/var/lib/kubelet/pods/29abf51f-23ae-4337-9cc4-7275f7aacd44/volumes" Jul 07 18:13:24.959668 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:24.959623 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx"] Jul 07 18:13:24.960190 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:24.959997 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29abf51f-23ae-4337-9cc4-7275f7aacd44" containerName="maas-api" Jul 07 18:13:24.960190 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:24.960008 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="29abf51f-23ae-4337-9cc4-7275f7aacd44" containerName="maas-api" Jul 07 18:13:24.960190 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:24.960097 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="29abf51f-23ae-4337-9cc4-7275f7aacd44" containerName="maas-api" Jul 07 18:13:24.963280 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:24.963258 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:13:24.965899 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:24.965865 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-19c08a-serving-cert\"" Jul 07 18:13:24.973451 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:24.973417 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx"] Jul 07 18:13:25.114297 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:25.114253 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-openshift-service-ca\") pod \"maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx\" (UID: \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\") " pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:13:25.114297 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:25.114305 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-maas-api-tls\") pod \"maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx\" (UID: \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\") " pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:13:25.114535 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:25.114432 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz22m\" (UniqueName: \"kubernetes.io/projected/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-kube-api-access-jz22m\") pod \"maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx\" (UID: \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\") " pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:13:25.216034 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:25.215923 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-openshift-service-ca\") pod \"maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx\" (UID: \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\") " pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:13:25.216034 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:25.215983 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-maas-api-tls\") pod \"maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx\" (UID: \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\") " pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:13:25.216332 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:25.216072 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz22m\" (UniqueName: \"kubernetes.io/projected/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-kube-api-access-jz22m\") pod \"maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx\" (UID: \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\") " pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:13:25.216332 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:13:25.216154 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-19c08a-serving-cert: secret "maas-api-e2e-inf-a-19c08a-serving-cert" not found Jul 07 18:13:25.216332 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:13:25.216249 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-maas-api-tls podName:38f6b9d3-ea8d-4008-8820-6ea60e08cd20 nodeName:}" failed. No retries permitted until 2026-07-07 18:13:25.716226043 +0000 UTC m=+2009.755552925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-maas-api-tls") pod "maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" (UID: "38f6b9d3-ea8d-4008-8820-6ea60e08cd20") : secret "maas-api-e2e-inf-a-19c08a-serving-cert" not found Jul 07 18:13:25.216729 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:25.216706 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-openshift-service-ca\") pod \"maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx\" (UID: \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\") " pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:13:25.225322 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:25.225277 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz22m\" (UniqueName: \"kubernetes.io/projected/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-kube-api-access-jz22m\") pod \"maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx\" (UID: \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\") " pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:13:25.721287 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:25.721242 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-maas-api-tls\") pod \"maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx\" (UID: \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\") " pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:13:25.724304 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:25.724269 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-maas-api-tls\") pod \"maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx\" (UID: \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\") " pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:13:25.877232 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:25.877191 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:13:26.031008 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:26.030970 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx"] Jul 07 18:13:26.036268 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:13:26.033246 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38f6b9d3_ea8d_4008_8820_6ea60e08cd20.slice/crio-74a2db7b664f856fed7fe18209f34ab85ffd5ca6b76e5b7ce14941e609963477 WatchSource:0}: Error finding container 74a2db7b664f856fed7fe18209f34ab85ffd5ca6b76e5b7ce14941e609963477: Status 404 returned error can't find the container with id 74a2db7b664f856fed7fe18209f34ab85ffd5ca6b76e5b7ce14941e609963477 Jul 07 18:13:26.037423 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:26.037398 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 18:13:26.236782 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:26.236685 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" event={"ID":"38f6b9d3-ea8d-4008-8820-6ea60e08cd20","Type":"ContainerStarted","Data":"74a2db7b664f856fed7fe18209f34ab85ffd5ca6b76e5b7ce14941e609963477"} Jul 07 18:13:27.243608 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:27.243567 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" event={"ID":"38f6b9d3-ea8d-4008-8820-6ea60e08cd20","Type":"ContainerStarted","Data":"01822dca3f9bddcb6ad904ef2f1ba329474a6be60c44259b2bea435e8f08260c"} Jul 07 18:13:27.244081 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:27.243665 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:13:27.262829 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:27.262769 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" podStartSLOduration=2.652793971 podStartE2EDuration="3.262737392s" totalImagesPullingTime="609.943421ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:13:27.261009562 +0000 UTC m=+2011.300336461" watchObservedRunningTime="2026-07-07 18:13:27.262737392 +0000 UTC m=+2011.302064329" Jul 07 18:13:33.034153 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.034114 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls"] Jul 07 18:13:33.037884 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.037863 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:13:33.040277 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.040244 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-ba4c79-serving-cert\"" Jul 07 18:13:33.048141 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.048107 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls"] Jul 07 18:13:33.192425 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.192382 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f64e72e4-965d-4311-b5ca-7a35a61f5046-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls\" (UID: \"f64e72e4-965d-4311-b5ca-7a35a61f5046\") " pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:13:33.192622 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.192497 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdw95\" (UniqueName: \"kubernetes.io/projected/f64e72e4-965d-4311-b5ca-7a35a61f5046-kube-api-access-mdw95\") pod \"maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls\" (UID: \"f64e72e4-965d-4311-b5ca-7a35a61f5046\") " pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:13:33.192622 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.192532 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f64e72e4-965d-4311-b5ca-7a35a61f5046-maas-api-tls\") pod \"maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls\" (UID: \"f64e72e4-965d-4311-b5ca-7a35a61f5046\") " pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:13:33.254916 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.254887 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:13:33.293568 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.293465 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f64e72e4-965d-4311-b5ca-7a35a61f5046-maas-api-tls\") pod \"maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls\" (UID: \"f64e72e4-965d-4311-b5ca-7a35a61f5046\") " pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:13:33.293568 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.293539 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f64e72e4-965d-4311-b5ca-7a35a61f5046-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls\" (UID: \"f64e72e4-965d-4311-b5ca-7a35a61f5046\") " pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:13:33.293821 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.293650 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdw95\" (UniqueName: \"kubernetes.io/projected/f64e72e4-965d-4311-b5ca-7a35a61f5046-kube-api-access-mdw95\") pod \"maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls\" (UID: \"f64e72e4-965d-4311-b5ca-7a35a61f5046\") " pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:13:33.293961 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:13:33.293937 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-ba4c79-serving-cert: secret "maas-api-e2e-inf-b-ba4c79-serving-cert" not found Jul 07 18:13:33.294033 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:13:33.294019 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f64e72e4-965d-4311-b5ca-7a35a61f5046-maas-api-tls podName:f64e72e4-965d-4311-b5ca-7a35a61f5046 nodeName:}" failed. No retries permitted until 2026-07-07 18:13:33.794002718 +0000 UTC m=+2017.833329598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f64e72e4-965d-4311-b5ca-7a35a61f5046-maas-api-tls") pod "maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" (UID: "f64e72e4-965d-4311-b5ca-7a35a61f5046") : secret "maas-api-e2e-inf-b-ba4c79-serving-cert" not found Jul 07 18:13:33.294393 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.294368 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f64e72e4-965d-4311-b5ca-7a35a61f5046-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls\" (UID: \"f64e72e4-965d-4311-b5ca-7a35a61f5046\") " pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:13:33.305355 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.305322 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdw95\" (UniqueName: \"kubernetes.io/projected/f64e72e4-965d-4311-b5ca-7a35a61f5046-kube-api-access-mdw95\") pod \"maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls\" (UID: \"f64e72e4-965d-4311-b5ca-7a35a61f5046\") " pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:13:33.797216 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.797173 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f64e72e4-965d-4311-b5ca-7a35a61f5046-maas-api-tls\") pod \"maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls\" (UID: \"f64e72e4-965d-4311-b5ca-7a35a61f5046\") " pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:13:33.799863 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.799830 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f64e72e4-965d-4311-b5ca-7a35a61f5046-maas-api-tls\") pod \"maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls\" (UID: \"f64e72e4-965d-4311-b5ca-7a35a61f5046\") " pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:13:33.952923 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:33.952880 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:13:34.108960 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:34.108931 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls"] Jul 07 18:13:34.111165 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:13:34.111133 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf64e72e4_965d_4311_b5ca_7a35a61f5046.slice/crio-a431f8988b838007602197878fad691607601e81271a685f5c011fd4e55bcd9a WatchSource:0}: Error finding container a431f8988b838007602197878fad691607601e81271a685f5c011fd4e55bcd9a: Status 404 returned error can't find the container with id a431f8988b838007602197878fad691607601e81271a685f5c011fd4e55bcd9a Jul 07 18:13:34.286072 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:34.285985 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" event={"ID":"f64e72e4-965d-4311-b5ca-7a35a61f5046","Type":"ContainerStarted","Data":"a431f8988b838007602197878fad691607601e81271a685f5c011fd4e55bcd9a"} Jul 07 18:13:35.293699 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:35.293661 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" event={"ID":"f64e72e4-965d-4311-b5ca-7a35a61f5046","Type":"ContainerStarted","Data":"14412b872f85ed03816ca2400ef5858816edfe2a26f68d69aa217b6ab469ddc3"} Jul 07 18:13:35.294212 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:35.293784 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:13:35.312166 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:35.312108 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" podStartSLOduration=1.925267271 podStartE2EDuration="2.312090165s" totalImagesPullingTime="386.822894ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:13:35.310825491 +0000 UTC m=+2019.350152389" watchObservedRunningTime="2026-07-07 18:13:35.312090165 +0000 UTC m=+2019.351417068" Jul 07 18:13:41.305304 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:13:41.305263 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:14:32.768946 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:32.768853 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx"] Jul 07 18:14:32.769516 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:32.769098 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" podUID="38f6b9d3-ea8d-4008-8820-6ea60e08cd20" containerName="maas-api" containerID="cri-o://01822dca3f9bddcb6ad904ef2f1ba329474a6be60c44259b2bea435e8f08260c" gracePeriod=30 Jul 07 18:14:33.024462 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.024403 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:14:33.039637 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.039607 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-openshift-service-ca\") pod \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\" (UID: \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\") " Jul 07 18:14:33.039775 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.039645 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jz22m\" (UniqueName: \"kubernetes.io/projected/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-kube-api-access-jz22m\") pod \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\" (UID: \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\") " Jul 07 18:14:33.039775 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.039707 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-maas-api-tls\") pod \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\" (UID: \"38f6b9d3-ea8d-4008-8820-6ea60e08cd20\") " Jul 07 18:14:33.040131 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.040087 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "38f6b9d3-ea8d-4008-8820-6ea60e08cd20" (UID: "38f6b9d3-ea8d-4008-8820-6ea60e08cd20"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:14:33.042469 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.042428 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-kube-api-access-jz22m" (OuterVolumeSpecName: "kube-api-access-jz22m") pod "38f6b9d3-ea8d-4008-8820-6ea60e08cd20" (UID: "38f6b9d3-ea8d-4008-8820-6ea60e08cd20"). InnerVolumeSpecName "kube-api-access-jz22m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:14:33.042821 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.042792 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "38f6b9d3-ea8d-4008-8820-6ea60e08cd20" (UID: "38f6b9d3-ea8d-4008-8820-6ea60e08cd20"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:14:33.140819 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.140781 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-openshift-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:14:33.140819 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.140815 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz22m\" (UniqueName: \"kubernetes.io/projected/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-kube-api-access-jz22m\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:14:33.140819 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.140825 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38f6b9d3-ea8d-4008-8820-6ea60e08cd20-maas-api-tls\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:14:33.619618 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.619586 2577 generic.go:356] "Generic (PLEG): container finished" podID="38f6b9d3-ea8d-4008-8820-6ea60e08cd20" containerID="01822dca3f9bddcb6ad904ef2f1ba329474a6be60c44259b2bea435e8f08260c" exitCode=0 Jul 07 18:14:33.619816 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.619634 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" Jul 07 18:14:33.619816 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.619664 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" event={"ID":"38f6b9d3-ea8d-4008-8820-6ea60e08cd20","Type":"ContainerDied","Data":"01822dca3f9bddcb6ad904ef2f1ba329474a6be60c44259b2bea435e8f08260c"} Jul 07 18:14:33.619816 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.619723 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx" event={"ID":"38f6b9d3-ea8d-4008-8820-6ea60e08cd20","Type":"ContainerDied","Data":"74a2db7b664f856fed7fe18209f34ab85ffd5ca6b76e5b7ce14941e609963477"} Jul 07 18:14:33.619816 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.619746 2577 scope.go:117] "RemoveContainer" containerID="01822dca3f9bddcb6ad904ef2f1ba329474a6be60c44259b2bea435e8f08260c" Jul 07 18:14:33.630080 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.630024 2577 scope.go:117] "RemoveContainer" containerID="01822dca3f9bddcb6ad904ef2f1ba329474a6be60c44259b2bea435e8f08260c" Jul 07 18:14:33.630403 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:14:33.630382 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01822dca3f9bddcb6ad904ef2f1ba329474a6be60c44259b2bea435e8f08260c\": container with ID starting with 01822dca3f9bddcb6ad904ef2f1ba329474a6be60c44259b2bea435e8f08260c not found: ID does not exist" containerID="01822dca3f9bddcb6ad904ef2f1ba329474a6be60c44259b2bea435e8f08260c" Jul 07 18:14:33.630480 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.630418 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01822dca3f9bddcb6ad904ef2f1ba329474a6be60c44259b2bea435e8f08260c"} err="failed to get container status \"01822dca3f9bddcb6ad904ef2f1ba329474a6be60c44259b2bea435e8f08260c\": rpc error: code = NotFound desc = could not find container \"01822dca3f9bddcb6ad904ef2f1ba329474a6be60c44259b2bea435e8f08260c\": container with ID starting with 01822dca3f9bddcb6ad904ef2f1ba329474a6be60c44259b2bea435e8f08260c not found: ID does not exist" Jul 07 18:14:33.642497 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.642459 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx"] Jul 07 18:14:33.646715 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:33.646682 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-19c08a-97dc8485b-krkvx"] Jul 07 18:14:34.523287 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:34.523242 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38f6b9d3-ea8d-4008-8820-6ea60e08cd20" path="/var/lib/kubelet/pods/38f6b9d3-ea8d-4008-8820-6ea60e08cd20/volumes" Jul 07 18:14:56.640241 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:56.640203 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 18:14:56.644026 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:14:56.643989 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 18:15:15.839652 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:15.839609 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls"] Jul 07 18:15:15.840247 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:15.839899 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" podUID="f64e72e4-965d-4311-b5ca-7a35a61f5046" containerName="maas-api" containerID="cri-o://14412b872f85ed03816ca2400ef5858816edfe2a26f68d69aa217b6ab469ddc3" gracePeriod=30 Jul 07 18:15:16.096535 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.096457 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:15:16.239274 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.239232 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdw95\" (UniqueName: \"kubernetes.io/projected/f64e72e4-965d-4311-b5ca-7a35a61f5046-kube-api-access-mdw95\") pod \"f64e72e4-965d-4311-b5ca-7a35a61f5046\" (UID: \"f64e72e4-965d-4311-b5ca-7a35a61f5046\") " Jul 07 18:15:16.239489 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.239337 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f64e72e4-965d-4311-b5ca-7a35a61f5046-openshift-service-ca\") pod \"f64e72e4-965d-4311-b5ca-7a35a61f5046\" (UID: \"f64e72e4-965d-4311-b5ca-7a35a61f5046\") " Jul 07 18:15:16.239489 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.239368 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f64e72e4-965d-4311-b5ca-7a35a61f5046-maas-api-tls\") pod \"f64e72e4-965d-4311-b5ca-7a35a61f5046\" (UID: \"f64e72e4-965d-4311-b5ca-7a35a61f5046\") " Jul 07 18:15:16.239767 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.239732 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f64e72e4-965d-4311-b5ca-7a35a61f5046-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f64e72e4-965d-4311-b5ca-7a35a61f5046" (UID: "f64e72e4-965d-4311-b5ca-7a35a61f5046"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:15:16.241735 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.241697 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f64e72e4-965d-4311-b5ca-7a35a61f5046-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f64e72e4-965d-4311-b5ca-7a35a61f5046" (UID: "f64e72e4-965d-4311-b5ca-7a35a61f5046"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:15:16.241869 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.241745 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f64e72e4-965d-4311-b5ca-7a35a61f5046-kube-api-access-mdw95" (OuterVolumeSpecName: "kube-api-access-mdw95") pod "f64e72e4-965d-4311-b5ca-7a35a61f5046" (UID: "f64e72e4-965d-4311-b5ca-7a35a61f5046"). InnerVolumeSpecName "kube-api-access-mdw95". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:15:16.340926 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.340867 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f64e72e4-965d-4311-b5ca-7a35a61f5046-maas-api-tls\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:15:16.340926 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.340920 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdw95\" (UniqueName: \"kubernetes.io/projected/f64e72e4-965d-4311-b5ca-7a35a61f5046-kube-api-access-mdw95\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:15:16.340926 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.340930 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f64e72e4-965d-4311-b5ca-7a35a61f5046-openshift-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:15:16.872369 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.872253 2577 generic.go:356] "Generic (PLEG): container finished" podID="f64e72e4-965d-4311-b5ca-7a35a61f5046" containerID="14412b872f85ed03816ca2400ef5858816edfe2a26f68d69aa217b6ab469ddc3" exitCode=0 Jul 07 18:15:16.872369 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.872306 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" Jul 07 18:15:16.872369 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.872348 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" event={"ID":"f64e72e4-965d-4311-b5ca-7a35a61f5046","Type":"ContainerDied","Data":"14412b872f85ed03816ca2400ef5858816edfe2a26f68d69aa217b6ab469ddc3"} Jul 07 18:15:16.872871 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.872415 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls" event={"ID":"f64e72e4-965d-4311-b5ca-7a35a61f5046","Type":"ContainerDied","Data":"a431f8988b838007602197878fad691607601e81271a685f5c011fd4e55bcd9a"} Jul 07 18:15:16.872871 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.872439 2577 scope.go:117] "RemoveContainer" containerID="14412b872f85ed03816ca2400ef5858816edfe2a26f68d69aa217b6ab469ddc3" Jul 07 18:15:16.882645 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.882620 2577 scope.go:117] "RemoveContainer" containerID="14412b872f85ed03816ca2400ef5858816edfe2a26f68d69aa217b6ab469ddc3" Jul 07 18:15:16.883021 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:15:16.882998 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14412b872f85ed03816ca2400ef5858816edfe2a26f68d69aa217b6ab469ddc3\": container with ID starting with 14412b872f85ed03816ca2400ef5858816edfe2a26f68d69aa217b6ab469ddc3 not found: ID does not exist" containerID="14412b872f85ed03816ca2400ef5858816edfe2a26f68d69aa217b6ab469ddc3" Jul 07 18:15:16.883094 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.883033 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14412b872f85ed03816ca2400ef5858816edfe2a26f68d69aa217b6ab469ddc3"} err="failed to get container status \"14412b872f85ed03816ca2400ef5858816edfe2a26f68d69aa217b6ab469ddc3\": rpc error: code = NotFound desc = could not find container \"14412b872f85ed03816ca2400ef5858816edfe2a26f68d69aa217b6ab469ddc3\": container with ID starting with 14412b872f85ed03816ca2400ef5858816edfe2a26f68d69aa217b6ab469ddc3 not found: ID does not exist" Jul 07 18:15:16.888711 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.888675 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls"] Jul 07 18:15:16.892726 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:16.892689 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-ba4c79-75f476779f-kjjls"] Jul 07 18:15:18.518243 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:18.518202 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f64e72e4-965d-4311-b5ca-7a35a61f5046" path="/var/lib/kubelet/pods/f64e72e4-965d-4311-b5ca-7a35a61f5046/volumes" Jul 07 18:15:25.523005 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.522962 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp"] Jul 07 18:15:25.523592 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.523568 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f64e72e4-965d-4311-b5ca-7a35a61f5046" containerName="maas-api" Jul 07 18:15:25.523640 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.523597 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f64e72e4-965d-4311-b5ca-7a35a61f5046" containerName="maas-api" Jul 07 18:15:25.523640 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.523624 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38f6b9d3-ea8d-4008-8820-6ea60e08cd20" containerName="maas-api" Jul 07 18:15:25.523640 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.523634 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="38f6b9d3-ea8d-4008-8820-6ea60e08cd20" containerName="maas-api" Jul 07 18:15:25.523747 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.523736 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="38f6b9d3-ea8d-4008-8820-6ea60e08cd20" containerName="maas-api" Jul 07 18:15:25.523787 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.523751 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f64e72e4-965d-4311-b5ca-7a35a61f5046" containerName="maas-api" Jul 07 18:15:25.528795 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.528762 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:25.532431 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.531685 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-e88d61-serving-cert\"" Jul 07 18:15:25.539080 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.538244 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp"] Jul 07 18:15:25.624453 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.624404 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38164601-6040-4cf9-9d65-b71b44d5145d-openshift-service-ca\") pod \"maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp\" (UID: \"38164601-6040-4cf9-9d65-b71b44d5145d\") " pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:25.624453 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.624455 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38164601-6040-4cf9-9d65-b71b44d5145d-maas-api-tls\") pod \"maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp\" (UID: \"38164601-6040-4cf9-9d65-b71b44d5145d\") " pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:25.624700 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.624491 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q89bm\" (UniqueName: \"kubernetes.io/projected/38164601-6040-4cf9-9d65-b71b44d5145d-kube-api-access-q89bm\") pod \"maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp\" (UID: \"38164601-6040-4cf9-9d65-b71b44d5145d\") " pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:25.725578 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.725535 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38164601-6040-4cf9-9d65-b71b44d5145d-openshift-service-ca\") pod \"maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp\" (UID: \"38164601-6040-4cf9-9d65-b71b44d5145d\") " pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:25.725578 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.725582 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38164601-6040-4cf9-9d65-b71b44d5145d-maas-api-tls\") pod \"maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp\" (UID: \"38164601-6040-4cf9-9d65-b71b44d5145d\") " pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:25.725899 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.725616 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q89bm\" (UniqueName: \"kubernetes.io/projected/38164601-6040-4cf9-9d65-b71b44d5145d-kube-api-access-q89bm\") pod \"maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp\" (UID: \"38164601-6040-4cf9-9d65-b71b44d5145d\") " pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:25.725899 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:15:25.725759 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-e88d61-serving-cert: secret "maas-api-e2e-api-a-e88d61-serving-cert" not found Jul 07 18:15:25.725899 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:15:25.725875 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38164601-6040-4cf9-9d65-b71b44d5145d-maas-api-tls podName:38164601-6040-4cf9-9d65-b71b44d5145d nodeName:}" failed. No retries permitted until 2026-07-07 18:15:26.225850281 +0000 UTC m=+2130.265177162 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/38164601-6040-4cf9-9d65-b71b44d5145d-maas-api-tls") pod "maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" (UID: "38164601-6040-4cf9-9d65-b71b44d5145d") : secret "maas-api-e2e-api-a-e88d61-serving-cert" not found Jul 07 18:15:25.726295 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.726272 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38164601-6040-4cf9-9d65-b71b44d5145d-openshift-service-ca\") pod \"maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp\" (UID: \"38164601-6040-4cf9-9d65-b71b44d5145d\") " pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:25.736729 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:25.736693 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q89bm\" (UniqueName: \"kubernetes.io/projected/38164601-6040-4cf9-9d65-b71b44d5145d-kube-api-access-q89bm\") pod \"maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp\" (UID: \"38164601-6040-4cf9-9d65-b71b44d5145d\") " pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:26.229967 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:26.229923 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38164601-6040-4cf9-9d65-b71b44d5145d-maas-api-tls\") pod \"maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp\" (UID: \"38164601-6040-4cf9-9d65-b71b44d5145d\") " pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:26.232658 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:26.232630 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38164601-6040-4cf9-9d65-b71b44d5145d-maas-api-tls\") pod \"maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp\" (UID: \"38164601-6040-4cf9-9d65-b71b44d5145d\") " pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:26.451085 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:26.451011 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:26.635638 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:26.635611 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp"] Jul 07 18:15:26.637930 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:15:26.637893 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38164601_6040_4cf9_9d65_b71b44d5145d.slice/crio-5d834ed92d160064c384444be67af70fcb0a36531846ccea78da90fc284d2385 WatchSource:0}: Error finding container 5d834ed92d160064c384444be67af70fcb0a36531846ccea78da90fc284d2385: Status 404 returned error can't find the container with id 5d834ed92d160064c384444be67af70fcb0a36531846ccea78da90fc284d2385 Jul 07 18:15:26.927932 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:26.927830 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" event={"ID":"38164601-6040-4cf9-9d65-b71b44d5145d","Type":"ContainerStarted","Data":"5d834ed92d160064c384444be67af70fcb0a36531846ccea78da90fc284d2385"} Jul 07 18:15:27.934649 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:27.934611 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" event={"ID":"38164601-6040-4cf9-9d65-b71b44d5145d","Type":"ContainerStarted","Data":"a09b864ccf7f2385645b4be91fd57c39b15b48dff10bd41944070fba881cef6d"} Jul 07 18:15:27.935099 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:27.934688 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:27.953767 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:27.953695 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" podStartSLOduration=2.432134309 podStartE2EDuration="2.953675741s" totalImagesPullingTime="521.541432ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:15:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:15:27.953325587 +0000 UTC m=+2131.992652492" watchObservedRunningTime="2026-07-07 18:15:27.953675741 +0000 UTC m=+2131.993002644" Jul 07 18:15:33.560385 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:33.560345 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg"] Jul 07 18:15:33.564055 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:33.564005 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:33.567348 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:33.567322 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-9c804d-serving-cert\"" Jul 07 18:15:33.576254 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:33.576222 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg"] Jul 07 18:15:33.602963 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:33.602923 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05f39460-66b6-439e-8649-11126aefbf6c-maas-api-tls\") pod \"maas-api-e2e-api-b-9c804d-77fd549996-b5nkg\" (UID: \"05f39460-66b6-439e-8649-11126aefbf6c\") " pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:33.603174 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:33.602986 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f39460-66b6-439e-8649-11126aefbf6c-openshift-service-ca\") pod \"maas-api-e2e-api-b-9c804d-77fd549996-b5nkg\" (UID: \"05f39460-66b6-439e-8649-11126aefbf6c\") " pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:33.603174 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:33.603082 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw94l\" (UniqueName: \"kubernetes.io/projected/05f39460-66b6-439e-8649-11126aefbf6c-kube-api-access-qw94l\") pod \"maas-api-e2e-api-b-9c804d-77fd549996-b5nkg\" (UID: \"05f39460-66b6-439e-8649-11126aefbf6c\") " pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:33.704466 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:33.704426 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05f39460-66b6-439e-8649-11126aefbf6c-maas-api-tls\") pod \"maas-api-e2e-api-b-9c804d-77fd549996-b5nkg\" (UID: \"05f39460-66b6-439e-8649-11126aefbf6c\") " pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:33.704688 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:33.704508 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f39460-66b6-439e-8649-11126aefbf6c-openshift-service-ca\") pod \"maas-api-e2e-api-b-9c804d-77fd549996-b5nkg\" (UID: \"05f39460-66b6-439e-8649-11126aefbf6c\") " pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:33.704688 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:33.704542 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw94l\" (UniqueName: \"kubernetes.io/projected/05f39460-66b6-439e-8649-11126aefbf6c-kube-api-access-qw94l\") pod \"maas-api-e2e-api-b-9c804d-77fd549996-b5nkg\" (UID: \"05f39460-66b6-439e-8649-11126aefbf6c\") " pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:33.704688 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:15:33.704585 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-9c804d-serving-cert: secret "maas-api-e2e-api-b-9c804d-serving-cert" not found Jul 07 18:15:33.704688 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:15:33.704659 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05f39460-66b6-439e-8649-11126aefbf6c-maas-api-tls podName:05f39460-66b6-439e-8649-11126aefbf6c nodeName:}" failed. No retries permitted until 2026-07-07 18:15:34.204644024 +0000 UTC m=+2138.243970906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/05f39460-66b6-439e-8649-11126aefbf6c-maas-api-tls") pod "maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" (UID: "05f39460-66b6-439e-8649-11126aefbf6c") : secret "maas-api-e2e-api-b-9c804d-serving-cert" not found Jul 07 18:15:33.705276 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:33.705252 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f39460-66b6-439e-8649-11126aefbf6c-openshift-service-ca\") pod \"maas-api-e2e-api-b-9c804d-77fd549996-b5nkg\" (UID: \"05f39460-66b6-439e-8649-11126aefbf6c\") " pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:33.721889 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:33.721855 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw94l\" (UniqueName: \"kubernetes.io/projected/05f39460-66b6-439e-8649-11126aefbf6c-kube-api-access-qw94l\") pod \"maas-api-e2e-api-b-9c804d-77fd549996-b5nkg\" (UID: \"05f39460-66b6-439e-8649-11126aefbf6c\") " pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:33.946375 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:33.946347 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:34.209622 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:34.209513 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05f39460-66b6-439e-8649-11126aefbf6c-maas-api-tls\") pod \"maas-api-e2e-api-b-9c804d-77fd549996-b5nkg\" (UID: \"05f39460-66b6-439e-8649-11126aefbf6c\") " pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:34.212316 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:34.212279 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05f39460-66b6-439e-8649-11126aefbf6c-maas-api-tls\") pod \"maas-api-e2e-api-b-9c804d-77fd549996-b5nkg\" (UID: \"05f39460-66b6-439e-8649-11126aefbf6c\") " pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:34.480863 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:34.480765 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:34.633210 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:34.633179 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg"] Jul 07 18:15:34.635204 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:15:34.635174 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f39460_66b6_439e_8649_11126aefbf6c.slice/crio-fe3dc4d0e46027ab5fca439e1bf4028d85ad785bfa0ea79028a6afb6e2cfd7b9 WatchSource:0}: Error finding container fe3dc4d0e46027ab5fca439e1bf4028d85ad785bfa0ea79028a6afb6e2cfd7b9: Status 404 returned error can't find the container with id fe3dc4d0e46027ab5fca439e1bf4028d85ad785bfa0ea79028a6afb6e2cfd7b9 Jul 07 18:15:34.977366 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:34.977322 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" event={"ID":"05f39460-66b6-439e-8649-11126aefbf6c","Type":"ContainerStarted","Data":"fe3dc4d0e46027ab5fca439e1bf4028d85ad785bfa0ea79028a6afb6e2cfd7b9"} Jul 07 18:15:35.992383 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:35.992337 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" event={"ID":"05f39460-66b6-439e-8649-11126aefbf6c","Type":"ContainerStarted","Data":"3043f3886e08f44daa42b76568f6df87eee1a48ce257a974018182cf07c63624"} Jul 07 18:15:35.992843 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:35.992449 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:36.013284 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:36.013235 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" podStartSLOduration=2.58713668 podStartE2EDuration="3.013220179s" totalImagesPullingTime="426.083499ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:15:36.012976332 +0000 UTC m=+2140.052303235" watchObservedRunningTime="2026-07-07 18:15:36.013220179 +0000 UTC m=+2140.052547081" Jul 07 18:15:42.004484 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:42.004449 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:47.092501 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:47.092397 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp"] Jul 07 18:15:47.093084 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:47.092688 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" podUID="38164601-6040-4cf9-9d65-b71b44d5145d" containerName="maas-api" containerID="cri-o://a09b864ccf7f2385645b4be91fd57c39b15b48dff10bd41944070fba881cef6d" gracePeriod=30 Jul 07 18:15:47.352090 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:47.351976 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:47.435805 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:47.435769 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38164601-6040-4cf9-9d65-b71b44d5145d-maas-api-tls\") pod \"38164601-6040-4cf9-9d65-b71b44d5145d\" (UID: \"38164601-6040-4cf9-9d65-b71b44d5145d\") " Jul 07 18:15:47.436006 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:47.435855 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q89bm\" (UniqueName: \"kubernetes.io/projected/38164601-6040-4cf9-9d65-b71b44d5145d-kube-api-access-q89bm\") pod \"38164601-6040-4cf9-9d65-b71b44d5145d\" (UID: \"38164601-6040-4cf9-9d65-b71b44d5145d\") " Jul 07 18:15:47.436006 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:47.435888 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38164601-6040-4cf9-9d65-b71b44d5145d-openshift-service-ca\") pod \"38164601-6040-4cf9-9d65-b71b44d5145d\" (UID: \"38164601-6040-4cf9-9d65-b71b44d5145d\") " Jul 07 18:15:47.436352 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:47.436326 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38164601-6040-4cf9-9d65-b71b44d5145d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "38164601-6040-4cf9-9d65-b71b44d5145d" (UID: "38164601-6040-4cf9-9d65-b71b44d5145d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:15:47.438215 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:47.438181 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38164601-6040-4cf9-9d65-b71b44d5145d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "38164601-6040-4cf9-9d65-b71b44d5145d" (UID: "38164601-6040-4cf9-9d65-b71b44d5145d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:15:47.438324 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:47.438227 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38164601-6040-4cf9-9d65-b71b44d5145d-kube-api-access-q89bm" (OuterVolumeSpecName: "kube-api-access-q89bm") pod "38164601-6040-4cf9-9d65-b71b44d5145d" (UID: "38164601-6040-4cf9-9d65-b71b44d5145d"). InnerVolumeSpecName "kube-api-access-q89bm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:15:47.537254 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:47.537210 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38164601-6040-4cf9-9d65-b71b44d5145d-maas-api-tls\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:15:47.537254 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:47.537240 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q89bm\" (UniqueName: \"kubernetes.io/projected/38164601-6040-4cf9-9d65-b71b44d5145d-kube-api-access-q89bm\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:15:47.537254 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:47.537251 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38164601-6040-4cf9-9d65-b71b44d5145d-openshift-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:15:48.067744 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:48.067708 2577 generic.go:356] "Generic (PLEG): container finished" podID="38164601-6040-4cf9-9d65-b71b44d5145d" containerID="a09b864ccf7f2385645b4be91fd57c39b15b48dff10bd41944070fba881cef6d" exitCode=0 Jul 07 18:15:48.067949 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:48.067752 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" event={"ID":"38164601-6040-4cf9-9d65-b71b44d5145d","Type":"ContainerDied","Data":"a09b864ccf7f2385645b4be91fd57c39b15b48dff10bd41944070fba881cef6d"} Jul 07 18:15:48.067949 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:48.067771 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" Jul 07 18:15:48.067949 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:48.067820 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp" event={"ID":"38164601-6040-4cf9-9d65-b71b44d5145d","Type":"ContainerDied","Data":"5d834ed92d160064c384444be67af70fcb0a36531846ccea78da90fc284d2385"} Jul 07 18:15:48.067949 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:48.067842 2577 scope.go:117] "RemoveContainer" containerID="a09b864ccf7f2385645b4be91fd57c39b15b48dff10bd41944070fba881cef6d" Jul 07 18:15:48.077942 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:48.077920 2577 scope.go:117] "RemoveContainer" containerID="a09b864ccf7f2385645b4be91fd57c39b15b48dff10bd41944070fba881cef6d" Jul 07 18:15:48.078286 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:15:48.078264 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a09b864ccf7f2385645b4be91fd57c39b15b48dff10bd41944070fba881cef6d\": container with ID starting with a09b864ccf7f2385645b4be91fd57c39b15b48dff10bd41944070fba881cef6d not found: ID does not exist" containerID="a09b864ccf7f2385645b4be91fd57c39b15b48dff10bd41944070fba881cef6d" Jul 07 18:15:48.078374 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:48.078300 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a09b864ccf7f2385645b4be91fd57c39b15b48dff10bd41944070fba881cef6d"} err="failed to get container status \"a09b864ccf7f2385645b4be91fd57c39b15b48dff10bd41944070fba881cef6d\": rpc error: code = NotFound desc = could not find container \"a09b864ccf7f2385645b4be91fd57c39b15b48dff10bd41944070fba881cef6d\": container with ID starting with a09b864ccf7f2385645b4be91fd57c39b15b48dff10bd41944070fba881cef6d not found: ID does not exist" Jul 07 18:15:48.097152 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:48.097116 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp"] Jul 07 18:15:48.108639 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:48.108604 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-e88d61-6b5557689c-mtdpp"] Jul 07 18:15:48.516786 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:48.516749 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38164601-6040-4cf9-9d65-b71b44d5145d" path="/var/lib/kubelet/pods/38164601-6040-4cf9-9d65-b71b44d5145d/volumes" Jul 07 18:15:54.353764 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:54.353718 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg"] Jul 07 18:15:54.354305 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:54.354116 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" podUID="05f39460-66b6-439e-8649-11126aefbf6c" containerName="maas-api" containerID="cri-o://3043f3886e08f44daa42b76568f6df87eee1a48ce257a974018182cf07c63624" gracePeriod=30 Jul 07 18:15:54.621617 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:54.621582 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:54.703794 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:54.703753 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05f39460-66b6-439e-8649-11126aefbf6c-maas-api-tls\") pod \"05f39460-66b6-439e-8649-11126aefbf6c\" (UID: \"05f39460-66b6-439e-8649-11126aefbf6c\") " Jul 07 18:15:54.703794 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:54.703798 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f39460-66b6-439e-8649-11126aefbf6c-openshift-service-ca\") pod \"05f39460-66b6-439e-8649-11126aefbf6c\" (UID: \"05f39460-66b6-439e-8649-11126aefbf6c\") " Jul 07 18:15:54.704130 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:54.703851 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw94l\" (UniqueName: \"kubernetes.io/projected/05f39460-66b6-439e-8649-11126aefbf6c-kube-api-access-qw94l\") pod \"05f39460-66b6-439e-8649-11126aefbf6c\" (UID: \"05f39460-66b6-439e-8649-11126aefbf6c\") " Jul 07 18:15:54.704328 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:54.704299 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05f39460-66b6-439e-8649-11126aefbf6c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "05f39460-66b6-439e-8649-11126aefbf6c" (UID: "05f39460-66b6-439e-8649-11126aefbf6c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:15:54.706381 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:54.706352 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05f39460-66b6-439e-8649-11126aefbf6c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "05f39460-66b6-439e-8649-11126aefbf6c" (UID: "05f39460-66b6-439e-8649-11126aefbf6c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:15:54.706454 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:54.706414 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05f39460-66b6-439e-8649-11126aefbf6c-kube-api-access-qw94l" (OuterVolumeSpecName: "kube-api-access-qw94l") pod "05f39460-66b6-439e-8649-11126aefbf6c" (UID: "05f39460-66b6-439e-8649-11126aefbf6c"). InnerVolumeSpecName "kube-api-access-qw94l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:15:54.805265 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:54.805218 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw94l\" (UniqueName: \"kubernetes.io/projected/05f39460-66b6-439e-8649-11126aefbf6c-kube-api-access-qw94l\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:15:54.805265 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:54.805258 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05f39460-66b6-439e-8649-11126aefbf6c-maas-api-tls\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:15:54.805265 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:54.805268 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05f39460-66b6-439e-8649-11126aefbf6c-openshift-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:15:55.108935 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:55.108894 2577 generic.go:356] "Generic (PLEG): container finished" podID="05f39460-66b6-439e-8649-11126aefbf6c" containerID="3043f3886e08f44daa42b76568f6df87eee1a48ce257a974018182cf07c63624" exitCode=0 Jul 07 18:15:55.109183 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:55.108946 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" Jul 07 18:15:55.109183 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:55.108979 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" event={"ID":"05f39460-66b6-439e-8649-11126aefbf6c","Type":"ContainerDied","Data":"3043f3886e08f44daa42b76568f6df87eee1a48ce257a974018182cf07c63624"} Jul 07 18:15:55.109183 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:55.109033 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg" event={"ID":"05f39460-66b6-439e-8649-11126aefbf6c","Type":"ContainerDied","Data":"fe3dc4d0e46027ab5fca439e1bf4028d85ad785bfa0ea79028a6afb6e2cfd7b9"} Jul 07 18:15:55.109183 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:55.109077 2577 scope.go:117] "RemoveContainer" containerID="3043f3886e08f44daa42b76568f6df87eee1a48ce257a974018182cf07c63624" Jul 07 18:15:55.119258 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:55.119233 2577 scope.go:117] "RemoveContainer" containerID="3043f3886e08f44daa42b76568f6df87eee1a48ce257a974018182cf07c63624" Jul 07 18:15:55.119609 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:15:55.119590 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3043f3886e08f44daa42b76568f6df87eee1a48ce257a974018182cf07c63624\": container with ID starting with 3043f3886e08f44daa42b76568f6df87eee1a48ce257a974018182cf07c63624 not found: ID does not exist" containerID="3043f3886e08f44daa42b76568f6df87eee1a48ce257a974018182cf07c63624" Jul 07 18:15:55.119658 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:55.119621 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3043f3886e08f44daa42b76568f6df87eee1a48ce257a974018182cf07c63624"} err="failed to get container status \"3043f3886e08f44daa42b76568f6df87eee1a48ce257a974018182cf07c63624\": rpc error: code = NotFound desc = could not find container \"3043f3886e08f44daa42b76568f6df87eee1a48ce257a974018182cf07c63624\": container with ID starting with 3043f3886e08f44daa42b76568f6df87eee1a48ce257a974018182cf07c63624 not found: ID does not exist" Jul 07 18:15:55.131312 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:55.131268 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg"] Jul 07 18:15:55.135142 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:55.135111 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-9c804d-77fd549996-b5nkg"] Jul 07 18:15:56.516966 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:15:56.516930 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05f39460-66b6-439e-8649-11126aefbf6c" path="/var/lib/kubelet/pods/05f39460-66b6-439e-8649-11126aefbf6c/volumes" Jul 07 18:16:03.764122 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.764061 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj"] Jul 07 18:16:03.765254 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.765223 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05f39460-66b6-439e-8649-11126aefbf6c" containerName="maas-api" Jul 07 18:16:03.765254 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.765255 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="05f39460-66b6-439e-8649-11126aefbf6c" containerName="maas-api" Jul 07 18:16:03.765409 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.765297 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38164601-6040-4cf9-9d65-b71b44d5145d" containerName="maas-api" Jul 07 18:16:03.765409 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.765306 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="38164601-6040-4cf9-9d65-b71b44d5145d" containerName="maas-api" Jul 07 18:16:03.765409 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.765405 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="38164601-6040-4cf9-9d65-b71b44d5145d" containerName="maas-api" Jul 07 18:16:03.765518 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.765419 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="05f39460-66b6-439e-8649-11126aefbf6c" containerName="maas-api" Jul 07 18:16:03.769068 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.769015 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:16:03.771309 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.771287 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-a68ed4-serving-cert\"" Jul 07 18:16:03.776897 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.776864 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj"] Jul 07 18:16:03.890472 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.890418 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdfkc\" (UniqueName: \"kubernetes.io/projected/898eb38f-3498-4339-86ec-d7903b51fbc0-kube-api-access-jdfkc\") pod \"maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj\" (UID: \"898eb38f-3498-4339-86ec-d7903b51fbc0\") " pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:16:03.890674 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.890492 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/898eb38f-3498-4339-86ec-d7903b51fbc0-maas-api-tls\") pod \"maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj\" (UID: \"898eb38f-3498-4339-86ec-d7903b51fbc0\") " pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:16:03.890674 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.890529 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/898eb38f-3498-4339-86ec-d7903b51fbc0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj\" (UID: \"898eb38f-3498-4339-86ec-d7903b51fbc0\") " pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:16:03.991860 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.991820 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/898eb38f-3498-4339-86ec-d7903b51fbc0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj\" (UID: \"898eb38f-3498-4339-86ec-d7903b51fbc0\") " pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:16:03.992100 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.991900 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdfkc\" (UniqueName: \"kubernetes.io/projected/898eb38f-3498-4339-86ec-d7903b51fbc0-kube-api-access-jdfkc\") pod \"maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj\" (UID: \"898eb38f-3498-4339-86ec-d7903b51fbc0\") " pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:16:03.992100 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.991936 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/898eb38f-3498-4339-86ec-d7903b51fbc0-maas-api-tls\") pod \"maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj\" (UID: \"898eb38f-3498-4339-86ec-d7903b51fbc0\") " pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:16:03.992100 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:16:03.992072 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-a68ed4-serving-cert: secret "maas-api-e2e-shared-a-a68ed4-serving-cert" not found Jul 07 18:16:03.992241 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:16:03.992134 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/898eb38f-3498-4339-86ec-d7903b51fbc0-maas-api-tls podName:898eb38f-3498-4339-86ec-d7903b51fbc0 nodeName:}" failed. No retries permitted until 2026-07-07 18:16:04.492114969 +0000 UTC m=+2168.531441869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/898eb38f-3498-4339-86ec-d7903b51fbc0-maas-api-tls") pod "maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" (UID: "898eb38f-3498-4339-86ec-d7903b51fbc0") : secret "maas-api-e2e-shared-a-a68ed4-serving-cert" not found Jul 07 18:16:03.992659 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:03.992637 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/898eb38f-3498-4339-86ec-d7903b51fbc0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj\" (UID: \"898eb38f-3498-4339-86ec-d7903b51fbc0\") " pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:16:04.001843 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:04.001783 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdfkc\" (UniqueName: \"kubernetes.io/projected/898eb38f-3498-4339-86ec-d7903b51fbc0-kube-api-access-jdfkc\") pod \"maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj\" (UID: \"898eb38f-3498-4339-86ec-d7903b51fbc0\") " pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:16:04.496247 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:04.496207 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/898eb38f-3498-4339-86ec-d7903b51fbc0-maas-api-tls\") pod \"maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj\" (UID: \"898eb38f-3498-4339-86ec-d7903b51fbc0\") " pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:16:04.499072 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:04.499008 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/898eb38f-3498-4339-86ec-d7903b51fbc0-maas-api-tls\") pod \"maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj\" (UID: \"898eb38f-3498-4339-86ec-d7903b51fbc0\") " pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:16:04.682957 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:04.682913 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:16:04.839811 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:04.839709 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj"] Jul 07 18:16:04.843020 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:16:04.842990 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod898eb38f_3498_4339_86ec_d7903b51fbc0.slice/crio-c03a5b8f2fa351da2e4aee64d23afbdb53534d1d770c56198e4ad070ecf2b1b7 WatchSource:0}: Error finding container c03a5b8f2fa351da2e4aee64d23afbdb53534d1d770c56198e4ad070ecf2b1b7: Status 404 returned error can't find the container with id c03a5b8f2fa351da2e4aee64d23afbdb53534d1d770c56198e4ad070ecf2b1b7 Jul 07 18:16:05.163415 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:05.163299 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" event={"ID":"898eb38f-3498-4339-86ec-d7903b51fbc0","Type":"ContainerStarted","Data":"c03a5b8f2fa351da2e4aee64d23afbdb53534d1d770c56198e4ad070ecf2b1b7"} Jul 07 18:16:06.171439 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:06.171396 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" event={"ID":"898eb38f-3498-4339-86ec-d7903b51fbc0","Type":"ContainerStarted","Data":"be235f938fd3e0d3dbecc1fb1d41ede2a02ec3d281400c8551cd2fcf2dce7053"} Jul 07 18:16:06.171905 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:06.171687 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:16:06.189599 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:06.189536 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" podStartSLOduration=2.831412292 podStartE2EDuration="3.18951533s" totalImagesPullingTime="358.103038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:16:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:16:06.186802905 +0000 UTC m=+2170.226129799" watchObservedRunningTime="2026-07-07 18:16:06.18951533 +0000 UTC m=+2170.228842233" Jul 07 18:16:11.924118 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:11.924075 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5"] Jul 07 18:16:11.928107 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:11.928076 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:16:11.930430 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:11.930399 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-bc5a1a-serving-cert\"" Jul 07 18:16:11.938739 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:11.938706 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5"] Jul 07 18:16:11.962307 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:11.962265 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfw9w\" (UniqueName: \"kubernetes.io/projected/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-kube-api-access-vfw9w\") pod \"maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5\" (UID: \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\") " pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:16:11.962492 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:11.962416 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5\" (UID: \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\") " pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:16:11.962542 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:11.962512 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-maas-api-tls\") pod \"maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5\" (UID: \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\") " pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:16:12.063576 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:12.063532 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-maas-api-tls\") pod \"maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5\" (UID: \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\") " pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:16:12.063576 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:12.063575 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfw9w\" (UniqueName: \"kubernetes.io/projected/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-kube-api-access-vfw9w\") pod \"maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5\" (UID: \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\") " pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:16:12.063853 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:12.063637 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5\" (UID: \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\") " pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:16:12.063853 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:16:12.063699 2577 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-bc5a1a-serving-cert: secret "maas-api-e2e-shared-b-bc5a1a-serving-cert" not found Jul 07 18:16:12.063853 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:16:12.063782 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-maas-api-tls podName:d1d6256b-b50e-4be8-b5a5-e337f9588ea0 nodeName:}" failed. No retries permitted until 2026-07-07 18:16:12.563759857 +0000 UTC m=+2176.603086742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-maas-api-tls") pod "maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" (UID: "d1d6256b-b50e-4be8-b5a5-e337f9588ea0") : secret "maas-api-e2e-shared-b-bc5a1a-serving-cert" not found Jul 07 18:16:12.064409 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:12.064388 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5\" (UID: \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\") " pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:16:12.075085 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:12.075028 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfw9w\" (UniqueName: \"kubernetes.io/projected/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-kube-api-access-vfw9w\") pod \"maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5\" (UID: \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\") " pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:16:12.182775 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:12.182736 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:16:12.568095 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:12.567951 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-maas-api-tls\") pod \"maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5\" (UID: \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\") " pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:16:12.570850 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:12.570812 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-maas-api-tls\") pod \"maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5\" (UID: \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\") " pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:16:12.840988 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:12.840884 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:16:13.027511 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:13.027480 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5"] Jul 07 18:16:13.030537 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:16:13.030492 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d6256b_b50e_4be8_b5a5_e337f9588ea0.slice/crio-9521a45683eb0afa2762df6ea7d88b9be81c93472b851afd45ec681f57fc2a98 WatchSource:0}: Error finding container 9521a45683eb0afa2762df6ea7d88b9be81c93472b851afd45ec681f57fc2a98: Status 404 returned error can't find the container with id 9521a45683eb0afa2762df6ea7d88b9be81c93472b851afd45ec681f57fc2a98 Jul 07 18:16:13.211645 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:13.211604 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" event={"ID":"d1d6256b-b50e-4be8-b5a5-e337f9588ea0","Type":"ContainerStarted","Data":"9521a45683eb0afa2762df6ea7d88b9be81c93472b851afd45ec681f57fc2a98"} Jul 07 18:16:14.219887 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:14.219842 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" event={"ID":"d1d6256b-b50e-4be8-b5a5-e337f9588ea0","Type":"ContainerStarted","Data":"10bffbe032b666ee8682f5ca165198896218662f6eb3e554dcc5b6baed9d6443"} Jul 07 18:16:14.220375 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:14.219950 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:16:14.240470 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:14.240410 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" podStartSLOduration=2.880489285 podStartE2EDuration="3.240390007s" totalImagesPullingTime="359.900722ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:16:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:16:14.237541341 +0000 UTC m=+2178.276868243" watchObservedRunningTime="2026-07-07 18:16:14.240390007 +0000 UTC m=+2178.279716909" Jul 07 18:16:20.230810 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:16:20.230776 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:19:56.680263 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:19:56.680232 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 18:19:56.690950 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:19:56.690915 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 18:21:54.362324 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:54.362226 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7445-140227099278784"] Jul 07 18:21:54.365859 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:54.365830 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227099278784" Jul 07 18:21:54.368562 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:54.368534 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8wxd5\"" Jul 07 18:21:54.373673 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:54.373644 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7445-140227099278784"] Jul 07 18:21:54.435869 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:54.435826 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffb6n\" (UniqueName: \"kubernetes.io/projected/feb316ce-807a-43ea-8bab-9660d354cc49-kube-api-access-ffb6n\") pod \"test-curl-7445-140227099278784\" (UID: \"feb316ce-807a-43ea-8bab-9660d354cc49\") " pod="opendatahub/test-curl-7445-140227099278784" Jul 07 18:21:54.536282 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:54.536243 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffb6n\" (UniqueName: \"kubernetes.io/projected/feb316ce-807a-43ea-8bab-9660d354cc49-kube-api-access-ffb6n\") pod \"test-curl-7445-140227099278784\" (UID: \"feb316ce-807a-43ea-8bab-9660d354cc49\") " pod="opendatahub/test-curl-7445-140227099278784" Jul 07 18:21:54.545576 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:54.545538 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffb6n\" (UniqueName: \"kubernetes.io/projected/feb316ce-807a-43ea-8bab-9660d354cc49-kube-api-access-ffb6n\") pod \"test-curl-7445-140227099278784\" (UID: \"feb316ce-807a-43ea-8bab-9660d354cc49\") " pod="opendatahub/test-curl-7445-140227099278784" Jul 07 18:21:54.678544 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:54.678509 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227099278784" Jul 07 18:21:54.830507 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:54.830478 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7445-140227099278784"] Jul 07 18:21:54.833618 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:21:54.833582 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfeb316ce_807a_43ea_8bab_9660d354cc49.slice/crio-b7326c4047954948eb6883dd802d0d3440d2c6ca8ad13fca5dae76de2ecd70d6 WatchSource:0}: Error finding container b7326c4047954948eb6883dd802d0d3440d2c6ca8ad13fca5dae76de2ecd70d6: Status 404 returned error can't find the container with id b7326c4047954948eb6883dd802d0d3440d2c6ca8ad13fca5dae76de2ecd70d6 Jul 07 18:21:54.834917 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:54.834896 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 18:21:55.114577 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:55.114481 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7445-140227099278784" event={"ID":"feb316ce-807a-43ea-8bab-9660d354cc49","Type":"ContainerStarted","Data":"b7326c4047954948eb6883dd802d0d3440d2c6ca8ad13fca5dae76de2ecd70d6"} Jul 07 18:21:56.121465 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:56.121423 2577 generic.go:356] "Generic (PLEG): container finished" podID="feb316ce-807a-43ea-8bab-9660d354cc49" containerID="d59558f9600f2695f1b79341039c332e52439c043d72d41431daa017fa0d726f" exitCode=0 Jul 07 18:21:56.121886 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:56.121479 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7445-140227099278784" event={"ID":"feb316ce-807a-43ea-8bab-9660d354cc49","Type":"ContainerDied","Data":"d59558f9600f2695f1b79341039c332e52439c043d72d41431daa017fa0d726f"} Jul 07 18:21:57.277486 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.277447 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227099278784" Jul 07 18:21:57.312602 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.312574 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7445-140227099278784_feb316ce-807a-43ea-8bab-9660d354cc49/test-curl-7445-140227099278784/0.log" Jul 07 18:21:57.335595 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.335549 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7445-140227099278784"] Jul 07 18:21:57.339530 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.339494 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7445-140227099278784"] Jul 07 18:21:57.364110 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.364076 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffb6n\" (UniqueName: \"kubernetes.io/projected/feb316ce-807a-43ea-8bab-9660d354cc49-kube-api-access-ffb6n\") pod \"feb316ce-807a-43ea-8bab-9660d354cc49\" (UID: \"feb316ce-807a-43ea-8bab-9660d354cc49\") " Jul 07 18:21:57.366483 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.366449 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feb316ce-807a-43ea-8bab-9660d354cc49-kube-api-access-ffb6n" (OuterVolumeSpecName: "kube-api-access-ffb6n") pod "feb316ce-807a-43ea-8bab-9660d354cc49" (UID: "feb316ce-807a-43ea-8bab-9660d354cc49"). InnerVolumeSpecName "kube-api-access-ffb6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:21:57.465662 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.465620 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffb6n\" (UniqueName: \"kubernetes.io/projected/feb316ce-807a-43ea-8bab-9660d354cc49-kube-api-access-ffb6n\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:21:57.480558 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.480514 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7445-140227099278896"] Jul 07 18:21:57.480920 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.480905 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feb316ce-807a-43ea-8bab-9660d354cc49" containerName="test-curl-7445-140227099278784" Jul 07 18:21:57.480964 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.480921 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="feb316ce-807a-43ea-8bab-9660d354cc49" containerName="test-curl-7445-140227099278784" Jul 07 18:21:57.481006 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.480997 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="feb316ce-807a-43ea-8bab-9660d354cc49" containerName="test-curl-7445-140227099278784" Jul 07 18:21:57.484198 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.484179 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227099278896" Jul 07 18:21:57.489721 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.489687 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7445-140227099278896"] Jul 07 18:21:57.566329 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.566281 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ffxg\" (UniqueName: \"kubernetes.io/projected/566f370e-f005-4348-a97b-3f2cb574918f-kube-api-access-9ffxg\") pod \"test-curl-7445-140227099278896\" (UID: \"566f370e-f005-4348-a97b-3f2cb574918f\") " pod="opendatahub/test-curl-7445-140227099278896" Jul 07 18:21:57.667409 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.667366 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ffxg\" (UniqueName: \"kubernetes.io/projected/566f370e-f005-4348-a97b-3f2cb574918f-kube-api-access-9ffxg\") pod \"test-curl-7445-140227099278896\" (UID: \"566f370e-f005-4348-a97b-3f2cb574918f\") " pod="opendatahub/test-curl-7445-140227099278896" Jul 07 18:21:57.676659 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.676620 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ffxg\" (UniqueName: \"kubernetes.io/projected/566f370e-f005-4348-a97b-3f2cb574918f-kube-api-access-9ffxg\") pod \"test-curl-7445-140227099278896\" (UID: \"566f370e-f005-4348-a97b-3f2cb574918f\") " pod="opendatahub/test-curl-7445-140227099278896" Jul 07 18:21:57.796437 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.796329 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227099278896" Jul 07 18:21:57.952274 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:57.952245 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7445-140227099278896"] Jul 07 18:21:57.954459 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:21:57.954433 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod566f370e_f005_4348_a97b_3f2cb574918f.slice/crio-1de3cce13f5f38cc2b164a68c318d3cca8a4195e27559bab46154c968632092f WatchSource:0}: Error finding container 1de3cce13f5f38cc2b164a68c318d3cca8a4195e27559bab46154c968632092f: Status 404 returned error can't find the container with id 1de3cce13f5f38cc2b164a68c318d3cca8a4195e27559bab46154c968632092f Jul 07 18:21:58.133442 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:58.133350 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227099278784" Jul 07 18:21:58.133442 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:58.133388 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7326c4047954948eb6883dd802d0d3440d2c6ca8ad13fca5dae76de2ecd70d6" Jul 07 18:21:58.134743 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:58.134714 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7445-140227099278896" event={"ID":"566f370e-f005-4348-a97b-3f2cb574918f","Type":"ContainerStarted","Data":"1de3cce13f5f38cc2b164a68c318d3cca8a4195e27559bab46154c968632092f"} Jul 07 18:21:58.144227 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:21:58.144186 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7445-140227099278784\" is forbidden: User \"system:node:ip-10-0-129-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-170.ec2.internal' and this object" podUID="feb316ce-807a-43ea-8bab-9660d354cc49" pod="opendatahub/test-curl-7445-140227099278784" Jul 07 18:21:58.517097 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:58.517035 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="feb316ce-807a-43ea-8bab-9660d354cc49" path="/var/lib/kubelet/pods/feb316ce-807a-43ea-8bab-9660d354cc49/volumes" Jul 07 18:21:59.142111 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:59.142074 2577 generic.go:356] "Generic (PLEG): container finished" podID="566f370e-f005-4348-a97b-3f2cb574918f" containerID="550add0b540e39113b2c9fd3a619c192b6d57caa3c975d71daa2c69983b1baa3" exitCode=0 Jul 07 18:21:59.142294 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:21:59.142120 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7445-140227099278896" event={"ID":"566f370e-f005-4348-a97b-3f2cb574918f","Type":"ContainerDied","Data":"550add0b540e39113b2c9fd3a619c192b6d57caa3c975d71daa2c69983b1baa3"} Jul 07 18:22:00.298466 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:00.298427 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227099278896" Jul 07 18:22:00.319088 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:00.319033 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7445-140227099278896_566f370e-f005-4348-a97b-3f2cb574918f/test-curl-7445-140227099278896/0.log" Jul 07 18:22:00.343784 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:00.343745 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7445-140227099278896"] Jul 07 18:22:00.346476 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:00.346434 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7445-140227099278896"] Jul 07 18:22:00.394930 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:00.394883 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ffxg\" (UniqueName: \"kubernetes.io/projected/566f370e-f005-4348-a97b-3f2cb574918f-kube-api-access-9ffxg\") pod \"566f370e-f005-4348-a97b-3f2cb574918f\" (UID: \"566f370e-f005-4348-a97b-3f2cb574918f\") " Jul 07 18:22:00.397277 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:00.397236 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/566f370e-f005-4348-a97b-3f2cb574918f-kube-api-access-9ffxg" (OuterVolumeSpecName: "kube-api-access-9ffxg") pod "566f370e-f005-4348-a97b-3f2cb574918f" (UID: "566f370e-f005-4348-a97b-3f2cb574918f"). InnerVolumeSpecName "kube-api-access-9ffxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:22:00.495786 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:00.495735 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ffxg\" (UniqueName: \"kubernetes.io/projected/566f370e-f005-4348-a97b-3f2cb574918f-kube-api-access-9ffxg\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:22:00.517210 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:00.517165 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="566f370e-f005-4348-a97b-3f2cb574918f" path="/var/lib/kubelet/pods/566f370e-f005-4348-a97b-3f2cb574918f/volumes" Jul 07 18:22:01.154951 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:01.154922 2577 scope.go:117] "RemoveContainer" containerID="550add0b540e39113b2c9fd3a619c192b6d57caa3c975d71daa2c69983b1baa3" Jul 07 18:22:01.155169 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:01.154962 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227099278896" Jul 07 18:22:03.820485 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:03.820429 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7445-140227095888784"] Jul 07 18:22:03.820875 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:03.820838 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="566f370e-f005-4348-a97b-3f2cb574918f" containerName="test-curl-7445-140227099278896" Jul 07 18:22:03.820875 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:03.820851 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="566f370e-f005-4348-a97b-3f2cb574918f" containerName="test-curl-7445-140227099278896" Jul 07 18:22:03.820962 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:03.820942 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="566f370e-f005-4348-a97b-3f2cb574918f" containerName="test-curl-7445-140227099278896" Jul 07 18:22:03.824018 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:03.823997 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227095888784" Jul 07 18:22:03.826449 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:03.826419 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8wxd5\"" Jul 07 18:22:03.831506 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:03.831476 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7445-140227095888784"] Jul 07 18:22:03.927945 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:03.927907 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f88c2\" (UniqueName: \"kubernetes.io/projected/7cb99c37-68ef-4a21-9d57-20d485c40b19-kube-api-access-f88c2\") pod \"test-curl-7445-140227095888784\" (UID: \"7cb99c37-68ef-4a21-9d57-20d485c40b19\") " pod="opendatahub/test-curl-7445-140227095888784" Jul 07 18:22:04.029210 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:04.029164 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f88c2\" (UniqueName: \"kubernetes.io/projected/7cb99c37-68ef-4a21-9d57-20d485c40b19-kube-api-access-f88c2\") pod \"test-curl-7445-140227095888784\" (UID: \"7cb99c37-68ef-4a21-9d57-20d485c40b19\") " pod="opendatahub/test-curl-7445-140227095888784" Jul 07 18:22:04.038417 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:04.038389 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f88c2\" (UniqueName: \"kubernetes.io/projected/7cb99c37-68ef-4a21-9d57-20d485c40b19-kube-api-access-f88c2\") pod \"test-curl-7445-140227095888784\" (UID: \"7cb99c37-68ef-4a21-9d57-20d485c40b19\") " pod="opendatahub/test-curl-7445-140227095888784" Jul 07 18:22:04.135232 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:04.135109 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227095888784" Jul 07 18:22:04.287574 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:04.287548 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7445-140227095888784"] Jul 07 18:22:04.289967 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:22:04.289934 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7cb99c37_68ef_4a21_9d57_20d485c40b19.slice/crio-c080d30a89225c18457c09e8bfb67a59abf5ec8f6b6e6dfeb600cd82a132c161 WatchSource:0}: Error finding container c080d30a89225c18457c09e8bfb67a59abf5ec8f6b6e6dfeb600cd82a132c161: Status 404 returned error can't find the container with id c080d30a89225c18457c09e8bfb67a59abf5ec8f6b6e6dfeb600cd82a132c161 Jul 07 18:22:05.180924 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:05.180883 2577 generic.go:356] "Generic (PLEG): container finished" podID="7cb99c37-68ef-4a21-9d57-20d485c40b19" containerID="120d01b4754710d03a37b689a0593d1c0e264f13cac6cedfbecb598bc30b7173" exitCode=0 Jul 07 18:22:05.181399 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:05.180980 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7445-140227095888784" event={"ID":"7cb99c37-68ef-4a21-9d57-20d485c40b19","Type":"ContainerDied","Data":"120d01b4754710d03a37b689a0593d1c0e264f13cac6cedfbecb598bc30b7173"} Jul 07 18:22:05.181399 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:05.181029 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7445-140227095888784" event={"ID":"7cb99c37-68ef-4a21-9d57-20d485c40b19","Type":"ContainerStarted","Data":"c080d30a89225c18457c09e8bfb67a59abf5ec8f6b6e6dfeb600cd82a132c161"} Jul 07 18:22:06.337948 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.337913 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227095888784" Jul 07 18:22:06.365437 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.365397 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7445-140227095888784_7cb99c37-68ef-4a21-9d57-20d485c40b19/test-curl-7445-140227095888784/0.log" Jul 07 18:22:06.390799 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.390755 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7445-140227095888784"] Jul 07 18:22:06.394606 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.394571 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7445-140227095888784"] Jul 07 18:22:06.449619 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.449529 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f88c2\" (UniqueName: \"kubernetes.io/projected/7cb99c37-68ef-4a21-9d57-20d485c40b19-kube-api-access-f88c2\") pod \"7cb99c37-68ef-4a21-9d57-20d485c40b19\" (UID: \"7cb99c37-68ef-4a21-9d57-20d485c40b19\") " Jul 07 18:22:06.451939 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.451892 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cb99c37-68ef-4a21-9d57-20d485c40b19-kube-api-access-f88c2" (OuterVolumeSpecName: "kube-api-access-f88c2") pod "7cb99c37-68ef-4a21-9d57-20d485c40b19" (UID: "7cb99c37-68ef-4a21-9d57-20d485c40b19"). InnerVolumeSpecName "kube-api-access-f88c2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:22:06.519193 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.519152 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cb99c37-68ef-4a21-9d57-20d485c40b19" path="/var/lib/kubelet/pods/7cb99c37-68ef-4a21-9d57-20d485c40b19/volumes" Jul 07 18:22:06.529994 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.529961 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7445-140227095889504"] Jul 07 18:22:06.530409 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.530394 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cb99c37-68ef-4a21-9d57-20d485c40b19" containerName="test-curl-7445-140227095888784" Jul 07 18:22:06.530470 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.530410 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cb99c37-68ef-4a21-9d57-20d485c40b19" containerName="test-curl-7445-140227095888784" Jul 07 18:22:06.530506 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.530473 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cb99c37-68ef-4a21-9d57-20d485c40b19" containerName="test-curl-7445-140227095888784" Jul 07 18:22:06.533645 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.533623 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227095889504" Jul 07 18:22:06.539877 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.539845 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7445-140227095889504"] Jul 07 18:22:06.550561 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.550527 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f88c2\" (UniqueName: \"kubernetes.io/projected/7cb99c37-68ef-4a21-9d57-20d485c40b19-kube-api-access-f88c2\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:22:06.652034 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.651990 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l7cq\" (UniqueName: \"kubernetes.io/projected/8791615a-8ae9-4404-904a-ded78f296797-kube-api-access-8l7cq\") pod \"test-curl-7445-140227095889504\" (UID: \"8791615a-8ae9-4404-904a-ded78f296797\") " pod="opendatahub/test-curl-7445-140227095889504" Jul 07 18:22:06.752763 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.752653 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l7cq\" (UniqueName: \"kubernetes.io/projected/8791615a-8ae9-4404-904a-ded78f296797-kube-api-access-8l7cq\") pod \"test-curl-7445-140227095889504\" (UID: \"8791615a-8ae9-4404-904a-ded78f296797\") " pod="opendatahub/test-curl-7445-140227095889504" Jul 07 18:22:06.762438 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.762406 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l7cq\" (UniqueName: \"kubernetes.io/projected/8791615a-8ae9-4404-904a-ded78f296797-kube-api-access-8l7cq\") pod \"test-curl-7445-140227095889504\" (UID: \"8791615a-8ae9-4404-904a-ded78f296797\") " pod="opendatahub/test-curl-7445-140227095889504" Jul 07 18:22:06.846115 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.846026 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227095889504" Jul 07 18:22:06.995947 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:06.995916 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7445-140227095889504"] Jul 07 18:22:06.998096 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:22:06.998060 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8791615a_8ae9_4404_904a_ded78f296797.slice/crio-86776c7270ebd907daf851837230b00192debc7810417748f47e9f83cdc035b5 WatchSource:0}: Error finding container 86776c7270ebd907daf851837230b00192debc7810417748f47e9f83cdc035b5: Status 404 returned error can't find the container with id 86776c7270ebd907daf851837230b00192debc7810417748f47e9f83cdc035b5 Jul 07 18:22:07.194269 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:07.194236 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227095888784" Jul 07 18:22:07.194269 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:07.194260 2577 scope.go:117] "RemoveContainer" containerID="120d01b4754710d03a37b689a0593d1c0e264f13cac6cedfbecb598bc30b7173" Jul 07 18:22:07.195759 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:07.195645 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7445-140227095889504" event={"ID":"8791615a-8ae9-4404-904a-ded78f296797","Type":"ContainerStarted","Data":"86776c7270ebd907daf851837230b00192debc7810417748f47e9f83cdc035b5"} Jul 07 18:22:08.205134 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:08.205035 2577 generic.go:356] "Generic (PLEG): container finished" podID="8791615a-8ae9-4404-904a-ded78f296797" containerID="f93c68ccae6ab2959a06d82387879e62522d09a92bd2e7088aa1fc38e0f48d51" exitCode=0 Jul 07 18:22:08.205604 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:08.205158 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7445-140227095889504" event={"ID":"8791615a-8ae9-4404-904a-ded78f296797","Type":"ContainerDied","Data":"f93c68ccae6ab2959a06d82387879e62522d09a92bd2e7088aa1fc38e0f48d51"} Jul 07 18:22:09.361565 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:09.361533 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227095889504" Jul 07 18:22:09.382426 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:09.382395 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7445-140227095889504_8791615a-8ae9-4404-904a-ded78f296797/test-curl-7445-140227095889504/0.log" Jul 07 18:22:09.409314 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:09.409274 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7445-140227095889504"] Jul 07 18:22:09.414984 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:09.414939 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7445-140227095889504"] Jul 07 18:22:09.480377 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:09.480281 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l7cq\" (UniqueName: \"kubernetes.io/projected/8791615a-8ae9-4404-904a-ded78f296797-kube-api-access-8l7cq\") pod \"8791615a-8ae9-4404-904a-ded78f296797\" (UID: \"8791615a-8ae9-4404-904a-ded78f296797\") " Jul 07 18:22:09.482639 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:09.482596 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8791615a-8ae9-4404-904a-ded78f296797-kube-api-access-8l7cq" (OuterVolumeSpecName: "kube-api-access-8l7cq") pod "8791615a-8ae9-4404-904a-ded78f296797" (UID: "8791615a-8ae9-4404-904a-ded78f296797"). InnerVolumeSpecName "kube-api-access-8l7cq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:22:09.581389 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:09.581260 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l7cq\" (UniqueName: \"kubernetes.io/projected/8791615a-8ae9-4404-904a-ded78f296797-kube-api-access-8l7cq\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:22:10.223382 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:10.223350 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7445-140227095889504" Jul 07 18:22:10.223576 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:10.223419 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86776c7270ebd907daf851837230b00192debc7810417748f47e9f83cdc035b5" Jul 07 18:22:10.233186 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:22:10.233151 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7445-140227095889504\" is forbidden: User \"system:node:ip-10-0-129-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-170.ec2.internal' and this object" podUID="8791615a-8ae9-4404-904a-ded78f296797" pod="opendatahub/test-curl-7445-140227095889504" Jul 07 18:22:10.516833 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:10.516744 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8791615a-8ae9-4404-904a-ded78f296797" path="/var/lib/kubelet/pods/8791615a-8ae9-4404-904a-ded78f296797/volumes" Jul 07 18:22:12.761074 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:12.761011 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj"] Jul 07 18:22:12.761523 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:12.761297 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" podUID="898eb38f-3498-4339-86ec-d7903b51fbc0" containerName="maas-api" containerID="cri-o://be235f938fd3e0d3dbecc1fb1d41ede2a02ec3d281400c8551cd2fcf2dce7053" gracePeriod=30 Jul 07 18:22:13.015521 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.015428 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:22:13.113833 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.113796 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/898eb38f-3498-4339-86ec-d7903b51fbc0-openshift-service-ca\") pod \"898eb38f-3498-4339-86ec-d7903b51fbc0\" (UID: \"898eb38f-3498-4339-86ec-d7903b51fbc0\") " Jul 07 18:22:13.113833 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.113839 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/898eb38f-3498-4339-86ec-d7903b51fbc0-maas-api-tls\") pod \"898eb38f-3498-4339-86ec-d7903b51fbc0\" (UID: \"898eb38f-3498-4339-86ec-d7903b51fbc0\") " Jul 07 18:22:13.114154 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.113942 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdfkc\" (UniqueName: \"kubernetes.io/projected/898eb38f-3498-4339-86ec-d7903b51fbc0-kube-api-access-jdfkc\") pod \"898eb38f-3498-4339-86ec-d7903b51fbc0\" (UID: \"898eb38f-3498-4339-86ec-d7903b51fbc0\") " Jul 07 18:22:13.114312 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.114288 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/898eb38f-3498-4339-86ec-d7903b51fbc0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "898eb38f-3498-4339-86ec-d7903b51fbc0" (UID: "898eb38f-3498-4339-86ec-d7903b51fbc0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:22:13.116322 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.116299 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/898eb38f-3498-4339-86ec-d7903b51fbc0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "898eb38f-3498-4339-86ec-d7903b51fbc0" (UID: "898eb38f-3498-4339-86ec-d7903b51fbc0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:22:13.116407 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.116338 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/898eb38f-3498-4339-86ec-d7903b51fbc0-kube-api-access-jdfkc" (OuterVolumeSpecName: "kube-api-access-jdfkc") pod "898eb38f-3498-4339-86ec-d7903b51fbc0" (UID: "898eb38f-3498-4339-86ec-d7903b51fbc0"). InnerVolumeSpecName "kube-api-access-jdfkc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:22:13.214915 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.214872 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdfkc\" (UniqueName: \"kubernetes.io/projected/898eb38f-3498-4339-86ec-d7903b51fbc0-kube-api-access-jdfkc\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:22:13.214915 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.214909 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/898eb38f-3498-4339-86ec-d7903b51fbc0-openshift-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:22:13.214915 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.214919 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/898eb38f-3498-4339-86ec-d7903b51fbc0-maas-api-tls\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:22:13.242108 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.242070 2577 generic.go:356] "Generic (PLEG): container finished" podID="898eb38f-3498-4339-86ec-d7903b51fbc0" containerID="be235f938fd3e0d3dbecc1fb1d41ede2a02ec3d281400c8551cd2fcf2dce7053" exitCode=0 Jul 07 18:22:13.242317 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.242143 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" Jul 07 18:22:13.242317 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.242082 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" event={"ID":"898eb38f-3498-4339-86ec-d7903b51fbc0","Type":"ContainerDied","Data":"be235f938fd3e0d3dbecc1fb1d41ede2a02ec3d281400c8551cd2fcf2dce7053"} Jul 07 18:22:13.242317 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.242209 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj" event={"ID":"898eb38f-3498-4339-86ec-d7903b51fbc0","Type":"ContainerDied","Data":"c03a5b8f2fa351da2e4aee64d23afbdb53534d1d770c56198e4ad070ecf2b1b7"} Jul 07 18:22:13.242317 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.242235 2577 scope.go:117] "RemoveContainer" containerID="be235f938fd3e0d3dbecc1fb1d41ede2a02ec3d281400c8551cd2fcf2dce7053" Jul 07 18:22:13.253495 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.253463 2577 scope.go:117] "RemoveContainer" containerID="be235f938fd3e0d3dbecc1fb1d41ede2a02ec3d281400c8551cd2fcf2dce7053" Jul 07 18:22:13.253883 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:22:13.253856 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be235f938fd3e0d3dbecc1fb1d41ede2a02ec3d281400c8551cd2fcf2dce7053\": container with ID starting with be235f938fd3e0d3dbecc1fb1d41ede2a02ec3d281400c8551cd2fcf2dce7053 not found: ID does not exist" containerID="be235f938fd3e0d3dbecc1fb1d41ede2a02ec3d281400c8551cd2fcf2dce7053" Jul 07 18:22:13.253993 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.253893 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be235f938fd3e0d3dbecc1fb1d41ede2a02ec3d281400c8551cd2fcf2dce7053"} err="failed to get container status \"be235f938fd3e0d3dbecc1fb1d41ede2a02ec3d281400c8551cd2fcf2dce7053\": rpc error: code = NotFound desc = could not find container \"be235f938fd3e0d3dbecc1fb1d41ede2a02ec3d281400c8551cd2fcf2dce7053\": container with ID starting with be235f938fd3e0d3dbecc1fb1d41ede2a02ec3d281400c8551cd2fcf2dce7053 not found: ID does not exist" Jul 07 18:22:13.264680 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.264639 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj"] Jul 07 18:22:13.268725 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:13.268633 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a68ed4-cd55bd8f6-p4cbj"] Jul 07 18:22:14.516359 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:14.516323 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="898eb38f-3498-4339-86ec-d7903b51fbc0" path="/var/lib/kubelet/pods/898eb38f-3498-4339-86ec-d7903b51fbc0/volumes" Jul 07 18:22:20.046276 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:20.046230 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5"] Jul 07 18:22:20.046777 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:20.046480 2577 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" podUID="d1d6256b-b50e-4be8-b5a5-e337f9588ea0" containerName="maas-api" containerID="cri-o://10bffbe032b666ee8682f5ca165198896218662f6eb3e554dcc5b6baed9d6443" gracePeriod=30 Jul 07 18:22:20.226156 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:20.226115 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" podUID="d1d6256b-b50e-4be8-b5a5-e337f9588ea0" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.64:8443/health\": dial tcp 10.132.0.64:8443: connect: connection refused" Jul 07 18:22:20.514280 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:20.514246 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:22:20.578993 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:20.578874 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfw9w\" (UniqueName: \"kubernetes.io/projected/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-kube-api-access-vfw9w\") pod \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\" (UID: \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\") " Jul 07 18:22:20.578993 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:20.578924 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-maas-api-tls\") pod \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\" (UID: \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\") " Jul 07 18:22:20.579283 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:20.579133 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-openshift-service-ca\") pod \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\" (UID: \"d1d6256b-b50e-4be8-b5a5-e337f9588ea0\") " Jul 07 18:22:20.579552 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:20.579524 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d1d6256b-b50e-4be8-b5a5-e337f9588ea0" (UID: "d1d6256b-b50e-4be8-b5a5-e337f9588ea0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:22:20.581493 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:20.581455 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-kube-api-access-vfw9w" (OuterVolumeSpecName: "kube-api-access-vfw9w") pod "d1d6256b-b50e-4be8-b5a5-e337f9588ea0" (UID: "d1d6256b-b50e-4be8-b5a5-e337f9588ea0"). InnerVolumeSpecName "kube-api-access-vfw9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:22:20.581627 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:20.581468 2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d1d6256b-b50e-4be8-b5a5-e337f9588ea0" (UID: "d1d6256b-b50e-4be8-b5a5-e337f9588ea0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:22:20.680129 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:20.680077 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfw9w\" (UniqueName: \"kubernetes.io/projected/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-kube-api-access-vfw9w\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:22:20.680129 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:20.680128 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-maas-api-tls\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:22:20.680129 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:20.680138 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d6256b-b50e-4be8-b5a5-e337f9588ea0-openshift-service-ca\") on node \"ip-10-0-129-170.ec2.internal\" DevicePath \"\"" Jul 07 18:22:21.303556 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:21.302509 2577 generic.go:356] "Generic (PLEG): container finished" podID="d1d6256b-b50e-4be8-b5a5-e337f9588ea0" containerID="10bffbe032b666ee8682f5ca165198896218662f6eb3e554dcc5b6baed9d6443" exitCode=0 Jul 07 18:22:21.303556 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:21.302626 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" event={"ID":"d1d6256b-b50e-4be8-b5a5-e337f9588ea0","Type":"ContainerDied","Data":"10bffbe032b666ee8682f5ca165198896218662f6eb3e554dcc5b6baed9d6443"} Jul 07 18:22:21.303556 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:21.302716 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" event={"ID":"d1d6256b-b50e-4be8-b5a5-e337f9588ea0","Type":"ContainerDied","Data":"9521a45683eb0afa2762df6ea7d88b9be81c93472b851afd45ec681f57fc2a98"} Jul 07 18:22:21.303556 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:21.302755 2577 scope.go:117] "RemoveContainer" containerID="10bffbe032b666ee8682f5ca165198896218662f6eb3e554dcc5b6baed9d6443" Jul 07 18:22:21.303556 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:21.302967 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5" Jul 07 18:22:21.313312 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:21.313285 2577 scope.go:117] "RemoveContainer" containerID="10bffbe032b666ee8682f5ca165198896218662f6eb3e554dcc5b6baed9d6443" Jul 07 18:22:21.313581 ip-10-0-129-170 kubenswrapper[2577]: E0707 18:22:21.313559 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10bffbe032b666ee8682f5ca165198896218662f6eb3e554dcc5b6baed9d6443\": container with ID starting with 10bffbe032b666ee8682f5ca165198896218662f6eb3e554dcc5b6baed9d6443 not found: ID does not exist" containerID="10bffbe032b666ee8682f5ca165198896218662f6eb3e554dcc5b6baed9d6443" Jul 07 18:22:21.313640 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:21.313592 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10bffbe032b666ee8682f5ca165198896218662f6eb3e554dcc5b6baed9d6443"} err="failed to get container status \"10bffbe032b666ee8682f5ca165198896218662f6eb3e554dcc5b6baed9d6443\": rpc error: code = NotFound desc = could not find container \"10bffbe032b666ee8682f5ca165198896218662f6eb3e554dcc5b6baed9d6443\": container with ID starting with 10bffbe032b666ee8682f5ca165198896218662f6eb3e554dcc5b6baed9d6443 not found: ID does not exist" Jul 07 18:22:21.326933 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:21.326895 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5"] Jul 07 18:22:21.331400 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:21.331367 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-bc5a1a-6f8448dbd8-s8xp5"] Jul 07 18:22:22.516377 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:22.516343 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1d6256b-b50e-4be8-b5a5-e337f9588ea0" path="/var/lib/kubelet/pods/d1d6256b-b50e-4be8-b5a5-e337f9588ea0/volumes" Jul 07 18:22:26.695527 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:26.695495 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67f644975-gc2wm_66e213cc-3700-4048-abba-f0ffc10c7116/authorino/0.log" Jul 07 18:22:32.044356 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:32.044313 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-65f5bb9c48-82jmb_0355921f-86fb-4ad0-ba9a-ead9a72baca8/maas-api/0.log" Jul 07 18:22:32.420933 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:32.420849 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76d699bd45-zgtps_89cadee5-d200-47b3-b0b8-05498c9fa8cf/manager/0.log" Jul 07 18:22:32.966982 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:32.966956 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f77d785b9-ffm2j_e53c694f-4f4f-4c73-8155-e3681092329d/copy-manifests/0.log" Jul 07 18:22:32.977381 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:32.977353 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f77d785b9-ffm2j_e53c694f-4f4f-4c73-8155-e3681092329d/manager/0.log" Jul 07 18:22:34.153632 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:34.153602 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76_c00d28e6-3bfc-4192-b25f-52868d7a2194/util/0.log" Jul 07 18:22:34.160597 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:34.160573 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76_c00d28e6-3bfc-4192-b25f-52868d7a2194/pull/0.log" Jul 07 18:22:34.167719 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:34.167693 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76_c00d28e6-3bfc-4192-b25f-52868d7a2194/extract/0.log" Jul 07 18:22:34.303239 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:34.303206 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs_51e7a8d5-d250-44cc-8187-b654dc0f4f3b/util/0.log" Jul 07 18:22:34.310427 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:34.310400 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs_51e7a8d5-d250-44cc-8187-b654dc0f4f3b/pull/0.log" Jul 07 18:22:34.316930 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:34.316908 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs_51e7a8d5-d250-44cc-8187-b654dc0f4f3b/extract/0.log" Jul 07 18:22:34.452319 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:34.452288 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76_b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d/extract/0.log" Jul 07 18:22:34.460349 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:34.460322 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76_b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d/util/0.log" Jul 07 18:22:34.467796 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:34.467767 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76_b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d/pull/0.log" Jul 07 18:22:34.588234 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:34.588204 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq_794029e6-db7c-4afd-b417-e66d0fdeb1df/pull/0.log" Jul 07 18:22:34.595133 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:34.595108 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq_794029e6-db7c-4afd-b417-e66d0fdeb1df/extract/0.log" Jul 07 18:22:34.601935 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:34.601913 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq_794029e6-db7c-4afd-b417-e66d0fdeb1df/util/0.log" Jul 07 18:22:34.735412 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:34.735325 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67f644975-gc2wm_66e213cc-3700-4048-abba-f0ffc10c7116/authorino/0.log" Jul 07 18:22:35.112119 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:35.112017 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-722dn_ff0b5bb9-13b4-4f08-80a0-02aeb4289c99/kuadrant-console-plugin/0.log" Jul 07 18:22:36.474353 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:36.474320 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5675bf5659-wf8n8_bd6fd06e-fe89-4331-9b94-409899ae1ce7/kube-auth-proxy/0.log" Jul 07 18:22:36.750925 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:36.750824 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-lh77w_d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe/payload-pre-processing/0.log" Jul 07 18:22:36.880029 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:36.879992 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-hrxjc_cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc/payload-processing/0.log" Jul 07 18:22:37.006599 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:37.006513 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c5c789b7f-26cs9_57a355c0-7af9-4412-b621-771fae3a122c/router/0.log" Jul 07 18:22:37.537128 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:37.537095 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-sn4b4_9e5876b7-495a-40a5-b519-93560475aef7/main/0.log" Jul 07 18:22:37.659941 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:37.659914 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-98w2k_fcbb157c-2999-4ec8-bb4c-d6b8b0c8a2b9/main/0.log" Jul 07 18:22:38.041414 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:38.041373 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-9f96l_61d2198e-3245-473d-9ee0-33e02270e9b0/main/0.log" Jul 07 18:22:45.661276 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:45.661239 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-q4psm_970a1d18-134b-417d-8b58-aa2ddd03d5bf/global-pull-secret-syncer/0.log" Jul 07 18:22:45.886809 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:45.886777 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qrfcq_cb7150e3-b3f8-4086-883c-2a2d4720246b/konnectivity-agent/0.log" Jul 07 18:22:45.908760 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:45.908728 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-170.ec2.internal_0973ca2b89d42fa29ef5474b01a07c7d/haproxy/0.log" Jul 07 18:22:50.151799 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:50.151767 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76_c00d28e6-3bfc-4192-b25f-52868d7a2194/extract/0.log" Jul 07 18:22:50.178365 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:50.178337 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76_c00d28e6-3bfc-4192-b25f-52868d7a2194/util/0.log" Jul 07 18:22:50.204214 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:50.204182 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f7m76_c00d28e6-3bfc-4192-b25f-52868d7a2194/pull/0.log" Jul 07 18:22:50.242363 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:50.242336 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs_51e7a8d5-d250-44cc-8187-b654dc0f4f3b/extract/0.log" Jul 07 18:22:50.268244 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:50.268211 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs_51e7a8d5-d250-44cc-8187-b654dc0f4f3b/util/0.log" Jul 07 18:22:50.294786 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:50.294758 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gsxxs_51e7a8d5-d250-44cc-8187-b654dc0f4f3b/pull/0.log" Jul 07 18:22:50.323845 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:50.323816 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76_b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d/extract/0.log" Jul 07 18:22:50.351116 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:50.351089 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76_b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d/util/0.log" Jul 07 18:22:50.374910 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:50.374822 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73f8g76_b06fb3cc-a5c4-4386-b5f1-18c1c2b9142d/pull/0.log" Jul 07 18:22:50.404617 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:50.404584 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq_794029e6-db7c-4afd-b417-e66d0fdeb1df/extract/0.log" Jul 07 18:22:50.428257 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:50.428232 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq_794029e6-db7c-4afd-b417-e66d0fdeb1df/util/0.log" Jul 07 18:22:50.455243 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:50.455215 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qqzfq_794029e6-db7c-4afd-b417-e66d0fdeb1df/pull/0.log" Jul 07 18:22:50.729687 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:50.729645 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-67f644975-gc2wm_66e213cc-3700-4048-abba-f0ffc10c7116/authorino/0.log" Jul 07 18:22:50.831304 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:50.831278 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-722dn_ff0b5bb9-13b4-4f08-80a0-02aeb4289c99/kuadrant-console-plugin/0.log" Jul 07 18:22:53.078775 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:53.078739 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-jzjvx_2d8f2aa1-a7fe-4b51-8e61-5eb356bd27e4/cluster-monitoring-operator/0.log" Jul 07 18:22:53.328292 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:53.328262 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfzd2_786621af-8860-4858-a4a8-f9bc5ed7c9fe/node-exporter/0.log" Jul 07 18:22:53.353424 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:53.353336 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfzd2_786621af-8860-4858-a4a8-f9bc5ed7c9fe/kube-rbac-proxy/0.log" Jul 07 18:22:53.382705 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:53.382682 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfzd2_786621af-8860-4858-a4a8-f9bc5ed7c9fe/init-textfile/0.log" Jul 07 18:22:55.791248 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.791220 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/2.log" Jul 07 18:22:55.795878 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.795848 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5brrn_b473a0e9-cbb7-404f-9f76-94e3bc7394ca/console-operator/3.log" Jul 07 18:22:55.914885 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.914847 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr"] Jul 07 18:22:55.915251 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.915237 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8791615a-8ae9-4404-904a-ded78f296797" containerName="test-curl-7445-140227095889504" Jul 07 18:22:55.915316 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.915253 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8791615a-8ae9-4404-904a-ded78f296797" containerName="test-curl-7445-140227095889504" Jul 07 18:22:55.915316 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.915266 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1d6256b-b50e-4be8-b5a5-e337f9588ea0" containerName="maas-api" Jul 07 18:22:55.915316 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.915274 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1d6256b-b50e-4be8-b5a5-e337f9588ea0" containerName="maas-api" Jul 07 18:22:55.915316 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.915292 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="898eb38f-3498-4339-86ec-d7903b51fbc0" containerName="maas-api" Jul 07 18:22:55.915316 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.915297 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="898eb38f-3498-4339-86ec-d7903b51fbc0" containerName="maas-api" Jul 07 18:22:55.915495 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.915355 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1d6256b-b50e-4be8-b5a5-e337f9588ea0" containerName="maas-api" Jul 07 18:22:55.915495 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.915364 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8791615a-8ae9-4404-904a-ded78f296797" containerName="test-curl-7445-140227095889504" Jul 07 18:22:55.915495 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.915371 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="898eb38f-3498-4339-86ec-d7903b51fbc0" containerName="maas-api" Jul 07 18:22:55.918795 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.918769 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:55.921342 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.921311 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6gdp2\"/\"default-dockercfg-scf4p\"" Jul 07 18:22:55.922161 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.922136 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6gdp2\"/\"openshift-service-ca.crt\"" Jul 07 18:22:55.922326 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.922137 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6gdp2\"/\"kube-root-ca.crt\"" Jul 07 18:22:55.927428 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:55.927400 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr"] Jul 07 18:22:56.012370 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.012311 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1da96b67-99f1-4424-8544-fb37ee51b52b-proc\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.012576 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.012392 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s24qd\" (UniqueName: \"kubernetes.io/projected/1da96b67-99f1-4424-8544-fb37ee51b52b-kube-api-access-s24qd\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.012576 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.012439 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1da96b67-99f1-4424-8544-fb37ee51b52b-sys\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.012576 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.012466 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1da96b67-99f1-4424-8544-fb37ee51b52b-podres\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.012576 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.012536 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1da96b67-99f1-4424-8544-fb37ee51b52b-lib-modules\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.113485 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.113378 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1da96b67-99f1-4424-8544-fb37ee51b52b-lib-modules\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.113485 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.113436 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1da96b67-99f1-4424-8544-fb37ee51b52b-proc\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.113485 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.113477 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s24qd\" (UniqueName: \"kubernetes.io/projected/1da96b67-99f1-4424-8544-fb37ee51b52b-kube-api-access-s24qd\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.113763 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.113512 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1da96b67-99f1-4424-8544-fb37ee51b52b-sys\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.113763 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.113538 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1da96b67-99f1-4424-8544-fb37ee51b52b-podres\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.113763 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.113606 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1da96b67-99f1-4424-8544-fb37ee51b52b-lib-modules\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.113763 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.113609 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1da96b67-99f1-4424-8544-fb37ee51b52b-proc\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.113763 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.113649 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1da96b67-99f1-4424-8544-fb37ee51b52b-podres\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.113763 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.113609 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1da96b67-99f1-4424-8544-fb37ee51b52b-sys\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.121604 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.121569 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s24qd\" (UniqueName: \"kubernetes.io/projected/1da96b67-99f1-4424-8544-fb37ee51b52b-kube-api-access-s24qd\") pod \"perf-node-gather-daemonset-lzppr\" (UID: \"1da96b67-99f1-4424-8544-fb37ee51b52b\") " pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.231184 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.231140 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.349811 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.349783 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-566b8fccc-cm2s9_c43a60e4-f75c-41d2-86a1-893e37287666/console/0.log" Jul 07 18:22:56.376911 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.376884 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr"] Jul 07 18:22:56.379088 ip-10-0-129-170 kubenswrapper[2577]: W0707 18:22:56.379034 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1da96b67_99f1_4424_8544_fb37ee51b52b.slice/crio-f6f1681a118926b4d876d33459c8ad00d683debf58c99243ecfd143469bcbeda WatchSource:0}: Error finding container f6f1681a118926b4d876d33459c8ad00d683debf58c99243ecfd143469bcbeda: Status 404 returned error can't find the container with id f6f1681a118926b4d876d33459c8ad00d683debf58c99243ecfd143469bcbeda Jul 07 18:22:56.408025 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.408000 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-dhg9p_ebf1176b-397e-426a-af38-040422cbb630/download-server/0.log" Jul 07 18:22:56.503225 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.503182 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" event={"ID":"1da96b67-99f1-4424-8544-fb37ee51b52b","Type":"ContainerStarted","Data":"e2e2beb303ccaa99a93373c959b4d8fcbc8e06b9205ea1442d08db0735fdfdaa"} Jul 07 18:22:56.503418 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.503249 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" event={"ID":"1da96b67-99f1-4424-8544-fb37ee51b52b","Type":"ContainerStarted","Data":"f6f1681a118926b4d876d33459c8ad00d683debf58c99243ecfd143469bcbeda"} Jul 07 18:22:56.503418 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.503303 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:22:56.520777 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:56.520722 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" podStartSLOduration=1.520704842 podStartE2EDuration="1.520704842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:22:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:22:56.519646733 +0000 UTC m=+2580.558973649" watchObservedRunningTime="2026-07-07 18:22:56.520704842 +0000 UTC m=+2580.560031745" Jul 07 18:22:57.006977 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:57.006944 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-s9wsz_344f0c5c-f855-4e9d-acf3-705194c9ade4/volume-data-source-validator/0.log" Jul 07 18:22:58.098103 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:58.098072 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b2xkh_8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71/dns/0.log" Jul 07 18:22:58.120103 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:58.120073 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b2xkh_8a1b3e2f-62eb-4dc7-8995-ca7ab8635f71/kube-rbac-proxy/0.log" Jul 07 18:22:58.221550 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:58.221506 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nck8x_d1d8d149-d9ff-4b3c-851f-ec9de53109b6/dns-node-resolver/0.log" Jul 07 18:22:58.859632 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:58.859605 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t5q9p_4db38d2b-e752-4fdf-81ed-d134f7831397/node-ca/0.log" Jul 07 18:22:59.931383 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:22:59.931356 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5675bf5659-wf8n8_bd6fd06e-fe89-4331-9b94-409899ae1ce7/kube-auth-proxy/0.log" Jul 07 18:23:00.061450 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:00.061405 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-lh77w_d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe/payload-pre-processing/0.log" Jul 07 18:23:00.113652 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:00.113620 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-hrxjc_cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc/payload-processing/0.log" Jul 07 18:23:00.138539 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:00.138503 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c5c789b7f-26cs9_57a355c0-7af9-4412-b621-771fae3a122c/router/0.log" Jul 07 18:23:00.885732 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:00.885705 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x5q45_258e1cc3-989f-4b11-9152-31b2aeb13bb9/serve-healthcheck-canary/0.log" Jul 07 18:23:01.352181 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:01.352148 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-2c8ld_055ae3eb-05b3-4509-9008-d62baf858fb3/insights-operator/0.log" Jul 07 18:23:01.354227 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:01.354205 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-2c8ld_055ae3eb-05b3-4509-9008-d62baf858fb3/insights-operator/1.log" Jul 07 18:23:01.454548 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:01.454516 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7j42m_5313c0af-946a-4e13-91f7-e2f1a5d05bb2/kube-rbac-proxy/0.log" Jul 07 18:23:01.476310 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:01.476284 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7j42m_5313c0af-946a-4e13-91f7-e2f1a5d05bb2/exporter/0.log" Jul 07 18:23:01.497578 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:01.497553 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7j42m_5313c0af-946a-4e13-91f7-e2f1a5d05bb2/extractor/0.log" Jul 07 18:23:02.520232 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:02.520201 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6gdp2/perf-node-gather-daemonset-lzppr" Jul 07 18:23:03.955301 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:03.955242 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-65f5bb9c48-82jmb_0355921f-86fb-4ad0-ba9a-ead9a72baca8/maas-api/0.log" Jul 07 18:23:04.108601 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:04.108547 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76d699bd45-zgtps_89cadee5-d200-47b3-b0b8-05498c9fa8cf/manager/0.log" Jul 07 18:23:04.315284 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:04.315199 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f77d785b9-ffm2j_e53c694f-4f4f-4c73-8155-e3681092329d/manager/0.log" Jul 07 18:23:04.336626 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:04.336600 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f77d785b9-ffm2j_e53c694f-4f4f-4c73-8155-e3681092329d/copy-manifests/0.log" Jul 07 18:23:06.006509 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:06.006476 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7789f458f-fzlt6_41d49a3e-d7ca-434d-9f37-6a310bdece6b/manager/0.log" Jul 07 18:23:06.529929 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:06.529900 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5675bf5659-wf8n8_bd6fd06e-fe89-4331-9b94-409899ae1ce7/kube-auth-proxy/0.log" Jul 07 18:23:06.636326 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:06.636287 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-lh77w_d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe/payload-pre-processing/0.log" Jul 07 18:23:06.679996 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:06.679961 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-hrxjc_cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc/payload-processing/0.log" Jul 07 18:23:06.695614 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:06.695584 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c5c789b7f-26cs9_57a355c0-7af9-4412-b621-771fae3a122c/router/0.log" Jul 07 18:23:08.037951 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:08.037832 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5675bf5659-wf8n8_bd6fd06e-fe89-4331-9b94-409899ae1ce7/kube-auth-proxy/0.log" Jul 07 18:23:08.142247 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:08.142212 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-lh77w_d9b3ea4b-48aa-4aab-9397-2024cb0f4bbe/payload-pre-processing/0.log" Jul 07 18:23:08.187580 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:08.187541 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-hrxjc_cfbe88c7-f58d-4e0f-bae2-bd9d95cbd9dc/payload-processing/0.log" Jul 07 18:23:08.204620 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:08.204587 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c5c789b7f-26cs9_57a355c0-7af9-4412-b621-771fae3a122c/router/0.log" Jul 07 18:23:11.617305 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:11.617268 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-dwjq4_55437412-4864-499e-82c2-a854b002bb47/kube-storage-version-migrator-operator/1.log" Jul 07 18:23:11.617963 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:11.617945 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-dwjq4_55437412-4864-499e-82c2-a854b002bb47/kube-storage-version-migrator-operator/0.log" Jul 07 18:23:12.626349 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:12.626265 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6vdhw_7b0b588c-d0f2-4e5f-9c0d-11e7e5a0fb71/kube-multus/0.log" Jul 07 18:23:12.968155 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:12.968128 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw276_557c3dc0-235b-4285-ad82-77d2fbbfa75f/kube-multus-additional-cni-plugins/0.log" Jul 07 18:23:12.992174 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:12.992144 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw276_557c3dc0-235b-4285-ad82-77d2fbbfa75f/egress-router-binary-copy/0.log" Jul 07 18:23:13.014096 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:13.014063 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw276_557c3dc0-235b-4285-ad82-77d2fbbfa75f/cni-plugins/0.log" Jul 07 18:23:13.035019 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:13.034994 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw276_557c3dc0-235b-4285-ad82-77d2fbbfa75f/bond-cni-plugin/0.log" Jul 07 18:23:13.059470 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:13.059439 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw276_557c3dc0-235b-4285-ad82-77d2fbbfa75f/routeoverride-cni/0.log" Jul 07 18:23:13.082188 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:13.082161 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw276_557c3dc0-235b-4285-ad82-77d2fbbfa75f/whereabouts-cni-bincopy/0.log" Jul 07 18:23:13.112240 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:13.112211 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw276_557c3dc0-235b-4285-ad82-77d2fbbfa75f/whereabouts-cni/0.log" Jul 07 18:23:13.278731 ip-10-0-129-170 kubenswrapper[2577]: I0707 18:23:13.278644 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jdmfc_9541dd58-70fc-4069-bdba-f4726c87e4d2/network-metrics-daemon/0.log"