Jul 05 00:09:21.935177 ip-10-0-130-0 systemd[1]: Starting Kubernetes Kubelet... Jul 05 00:09:22.380716 ip-10-0-130-0 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 00:09:22.380716 ip-10-0-130-0 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 05 00:09:22.380716 ip-10-0-130-0 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 00:09:22.380716 ip-10-0-130-0 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 05 00:09:22.380716 ip-10-0-130-0 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 00:09:22.382724 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.382643 2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 05 00:09:22.389551 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389536 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 00:09:22.389551 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389551 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389555 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389558 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389560 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389563 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389566 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389570 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389572 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389575 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389578 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389580 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389583 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389587 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389591 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389594 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389596 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389599 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389601 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389604 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389613 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 00:09:22.389616 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389617 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389619 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389622 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389625 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389627 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389630 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389633 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389635 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389638 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389640 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389643 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389646 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389648 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389651 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389654 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389656 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389659 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389661 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389664 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 00:09:22.390127 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389666 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389668 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389672 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389674 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389677 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389679 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389682 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389684 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389687 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389689 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389692 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389694 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389696 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389699 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389701 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389706 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389709 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389714 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389718 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389720 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 00:09:22.390584 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389724 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389726 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389729 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389732 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389734 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389737 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389739 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389741 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389744 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389746 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389749 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389752 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389754 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389757 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389759 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389761 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389764 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389766 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389769 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 00:09:22.391102 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389771 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389774 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389776 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389779 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389781 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389784 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389786 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389789 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389792 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389794 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389797 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389799 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389802 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389804 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389807 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389810 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389813 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389817 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389819 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.389822 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 00:09:22.391562 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390521 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390528 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390530 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390533 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390536 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390539 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390542 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390544 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390547 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390549 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390552 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390554 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390557 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390559 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390562 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390564 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390567 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390588 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390609 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390614 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 00:09:22.392057 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390619 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390623 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390626 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390629 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390632 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390635 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390637 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390640 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390642 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390645 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390649 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390651 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390654 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390657 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390659 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390662 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390664 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390667 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390669 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390672 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 00:09:22.392579 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390675 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390677 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390680 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390682 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390686 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390690 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390693 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390697 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390700 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390703 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390706 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390710 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390713 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390716 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390718 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390721 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390724 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390727 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390729 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 00:09:22.393086 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390732 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390735 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390738 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390740 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390745 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390749 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390752 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390754 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390758 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390761 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390764 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390767 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390769 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390771 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390774 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390776 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390779 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390781 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390784 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390786 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 00:09:22.393544 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390788 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390791 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390793 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390797 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390799 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390802 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390806 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390808 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390811 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390813 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390815 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390818 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390820 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390823 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390825 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390828 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390830 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390834 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390838 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.390841 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 00:09:22.394052 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391827 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391835 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391839 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391842 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391845 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391848 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391850 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391853 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391856 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391860 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391863 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391865 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391868 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391870 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391873 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391876 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391879 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391887 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391890 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 00:09:22.394571 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391892 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391895 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391897 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391918 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391920 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391923 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391926 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391928 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391930 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391934 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391937 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391940 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391942 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391945 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391947 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391950 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391953 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391955 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391957 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 00:09:22.395074 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391960 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391962 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391965 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391967 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391973 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391975 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391977 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391981 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391983 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391986 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391989 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.391991 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392000 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392002 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392005 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392007 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392009 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392012 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392015 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392017 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 00:09:22.395537 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392019 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392022 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392024 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392027 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392029 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392032 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392034 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392036 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392039 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392041 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392044 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392046 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392048 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392051 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392054 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392056 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392061 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392064 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392066 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392069 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 00:09:22.396085 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392071 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392074 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392077 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392079 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392083 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392092 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392094 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392097 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392099 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392102 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392104 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392107 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392109 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392112 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392114 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392117 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392119 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392122 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392124 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 00:09:22.396573 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392126 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392129 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392195 2558 flags.go:64] FLAG: --address="0.0.0.0" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392202 2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392211 2558 flags.go:64] FLAG: --anonymous-auth="true" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392215 2558 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392219 2558 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392222 2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392226 2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392231 2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392235 2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392238 2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392241 2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392244 2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392247 2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392250 2558 flags.go:64] FLAG: --cgroup-root="" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392253 2558 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392255 2558 flags.go:64] FLAG: --client-ca-file="" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392258 2558 flags.go:64] FLAG: --cloud-provider="external" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392269 2558 flags.go:64] FLAG: --cluster-dns="[]" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392275 2558 flags.go:64] FLAG: --cluster-domain="" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392278 2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392281 2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392284 2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 05 00:09:22.397045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392287 2558 flags.go:64] FLAG: --container-log-max-files="5" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392291 2558 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392293 2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392296 2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392300 2558 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392303 2558 flags.go:64] FLAG: --contention-profiling="false" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392305 2558 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392308 2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392311 2558 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392314 2558 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392318 2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392321 2558 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392324 2558 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392326 2558 flags.go:64] FLAG: --enable-load-reader="false" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392329 2558 flags.go:64] FLAG: --enable-server="true" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392332 2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392339 2558 flags.go:64] FLAG: --event-burst="100" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392343 2558 flags.go:64] FLAG: --event-qps="50" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392346 2558 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392348 2558 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392351 2558 flags.go:64] FLAG: --eviction-hard="" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392355 2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392357 2558 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392360 2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392363 2558 flags.go:64] FLAG: --eviction-soft="" Jul 05 00:09:22.397749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392366 2558 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392368 2558 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392371 2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392379 2558 flags.go:64] FLAG: --experimental-mounter-path="" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392382 2558 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392385 2558 flags.go:64] FLAG: --fail-swap-on="true" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392387 2558 flags.go:64] FLAG: --feature-gates="" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392391 2558 flags.go:64] FLAG: --file-check-frequency="20s" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392394 2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392397 2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392399 2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392402 2558 flags.go:64] FLAG: --healthz-port="10248" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392404 2558 flags.go:64] FLAG: --help="false" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392407 2558 flags.go:64] FLAG: --hostname-override="ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392410 2558 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392413 2558 flags.go:64] FLAG: --http-check-frequency="20s" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392415 2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392418 2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392421 2558 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392424 2558 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392427 2558 flags.go:64] FLAG: --image-service-endpoint="" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392430 2558 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392433 2558 flags.go:64] FLAG: --kube-api-burst="100" Jul 05 00:09:22.398383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392436 2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392440 2558 flags.go:64] FLAG: --kube-api-qps="50" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392443 2558 flags.go:64] FLAG: --kube-reserved="" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392445 2558 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392448 2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392451 2558 flags.go:64] FLAG: --kubelet-cgroups="" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392453 2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392456 2558 flags.go:64] FLAG: --lock-file="" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392459 2558 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392461 2558 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392464 2558 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392469 2558 flags.go:64] FLAG: --log-json-split-stream="false" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392472 2558 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392480 2558 flags.go:64] FLAG: --log-text-split-stream="false" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392483 2558 flags.go:64] FLAG: --logging-format="text" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392486 2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392489 2558 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392492 2558 flags.go:64] FLAG: --manifest-url="" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392494 2558 flags.go:64] FLAG: --manifest-url-header="" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392498 2558 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392501 2558 flags.go:64] FLAG: --max-open-files="1000000" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392506 2558 flags.go:64] FLAG: --max-pods="110" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392509 2558 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392511 2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392514 2558 flags.go:64] FLAG: --memory-manager-policy="None" Jul 05 00:09:22.399058 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392517 2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392520 2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392522 2558 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392525 2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392532 2558 flags.go:64] FLAG: --node-status-max-images="50" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392536 2558 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392539 2558 flags.go:64] FLAG: --oom-score-adj="-999" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392542 2558 flags.go:64] FLAG: --pod-cidr="" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392546 2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392551 2558 flags.go:64] FLAG: --pod-manifest-path="" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392553 2558 flags.go:64] FLAG: --pod-max-pids="-1" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392557 2558 flags.go:64] FLAG: --pods-per-core="0" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392559 2558 flags.go:64] FLAG: --port="10250" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392562 2558 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392565 2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0972e92b74b746882" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392568 2558 flags.go:64] FLAG: --qos-reserved="" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392571 2558 flags.go:64] FLAG: --read-only-port="10255" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392574 2558 flags.go:64] FLAG: --register-node="true" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392576 2558 flags.go:64] FLAG: --register-with-taints="" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392584 2558 flags.go:64] FLAG: --registry-burst="10" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392587 2558 flags.go:64] FLAG: --registry-qps="5" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392590 2558 flags.go:64] FLAG: --reserved-cpus="" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392593 2558 flags.go:64] FLAG: --reserved-memory="" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392596 2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392599 2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 05 00:09:22.400137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392602 2558 flags.go:64] FLAG: --rotate-certificates="false" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392605 2558 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392607 2558 flags.go:64] FLAG: --runonce="false" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392610 2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392613 2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392616 2558 flags.go:64] FLAG: --seccomp-default="false" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392619 2558 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392621 2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392624 2558 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392627 2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392630 2558 flags.go:64] FLAG: --storage-driver-password="root" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392632 2558 flags.go:64] FLAG: --storage-driver-secure="false" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392637 2558 flags.go:64] FLAG: --storage-driver-table="stats" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392639 2558 flags.go:64] FLAG: --storage-driver-user="root" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392642 2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392645 2558 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392648 2558 flags.go:64] FLAG: --system-cgroups="" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392651 2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392656 2558 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392659 2558 flags.go:64] FLAG: --tls-cert-file="" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392661 2558 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392667 2558 flags.go:64] FLAG: --tls-min-version="" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392670 2558 flags.go:64] FLAG: --tls-private-key-file="" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392673 2558 flags.go:64] FLAG: --topology-manager-policy="none" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392675 2558 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 05 00:09:22.401194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392678 2558 flags.go:64] FLAG: --topology-manager-scope="container" Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392681 2558 flags.go:64] FLAG: --v="2" Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392684 2558 flags.go:64] FLAG: --version="false" Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392688 2558 flags.go:64] FLAG: --vmodule="" Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392692 2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.392695 2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392834 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392838 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392841 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392844 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392847 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392849 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392852 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392854 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392857 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392859 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392861 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392864 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392866 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392869 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392872 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 00:09:22.401855 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392874 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392877 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392880 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392883 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392885 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392888 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392891 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392893 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392895 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392914 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392918 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392921 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392924 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392926 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392930 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392933 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392938 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392942 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392945 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 00:09:22.402686 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392948 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392951 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392954 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392957 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392959 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392962 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392965 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392967 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392970 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392973 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392975 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392979 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392982 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392985 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392987 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392991 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392993 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392996 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.392998 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393001 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 00:09:22.403529 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393003 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393006 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393008 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393011 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393013 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393016 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393018 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393021 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393023 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393026 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393029 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393031 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393034 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393036 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393039 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393042 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393044 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393046 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393049 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393051 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 00:09:22.404219 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393054 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393056 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393059 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393063 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393065 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393067 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393070 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393075 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393078 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393080 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393082 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393085 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393088 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393091 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393093 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393095 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393098 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393100 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393104 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393106 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 00:09:22.405061 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393109 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 00:09:22.405739 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393111 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 00:09:22.405739 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393114 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 00:09:22.405739 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393116 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 00:09:22.405739 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.393119 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 00:09:22.405739 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.393878 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 00:09:22.405739 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.401251 2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 05 00:09:22.405739 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.401270 2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 05 00:09:22.405739 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.402159 2558 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 05 00:09:22.405739 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.402173 2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 05 00:09:22.405739 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402257 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 00:09:22.405739 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402267 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 00:09:22.405739 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402274 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 00:09:22.405739 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402279 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 00:09:22.405739 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402284 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402288 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402293 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402297 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402301 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402306 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402310 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402315 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402319 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402323 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402327 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402331 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402335 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402339 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402343 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402347 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402351 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402355 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402359 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402364 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 00:09:22.406143 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402368 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402378 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402382 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402387 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402391 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402398 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402404 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402409 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402414 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402418 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402423 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402427 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402431 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402436 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402440 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402444 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402448 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402452 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402457 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402461 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 00:09:22.406743 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402465 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402469 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402473 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402477 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402481 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402486 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402490 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402494 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402498 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402502 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402506 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402510 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402514 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402518 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402523 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402528 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402532 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402537 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402541 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402545 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 00:09:22.407309 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402549 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402553 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402558 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402562 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402566 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402570 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402574 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402578 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402582 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402586 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402590 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402594 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402598 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402602 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402606 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402610 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402614 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402619 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402623 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402627 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 00:09:22.407844 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402631 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 00:09:22.408361 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402636 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 00:09:22.408361 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402640 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 00:09:22.408361 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402644 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 00:09:22.408361 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402648 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 00:09:22.408361 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402652 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 00:09:22.408361 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402656 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 00:09:22.408361 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402662 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 00:09:22.408361 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402668 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 00:09:22.408361 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402674 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 00:09:22.408361 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402678 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 00:09:22.408361 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402682 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 00:09:22.408361 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402686 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 00:09:22.408361 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402690 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 00:09:22.408361 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402694 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.402701 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402876 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402884 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402889 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402893 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402916 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402921 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402925 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402930 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402934 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402938 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402942 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402946 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402949 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 00:09:22.408754 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402953 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402958 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402965 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402970 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402974 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402979 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402983 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402987 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402991 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.402996 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403000 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403005 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403010 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403014 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403018 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403022 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403026 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403030 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403034 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403039 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 00:09:22.409150 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403043 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403047 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403051 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403055 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403059 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403063 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403067 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403070 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403074 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403079 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403083 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403087 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403091 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403098 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403103 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403108 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403112 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403116 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403121 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403125 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 00:09:22.409633 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403129 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403133 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403137 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403141 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403145 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403150 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403154 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403158 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403162 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403166 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403170 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403174 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403178 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403182 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403186 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403189 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403194 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403197 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403202 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403206 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 00:09:22.410182 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403210 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403216 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403221 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403226 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403231 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403235 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403239 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403244 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403248 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403253 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403257 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403262 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403266 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403271 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403276 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403280 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403284 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403289 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403293 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 00:09:22.410810 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403298 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 00:09:22.411588 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403302 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 00:09:22.411588 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403306 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 00:09:22.411588 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403310 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 00:09:22.411588 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403314 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 00:09:22.411588 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403318 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 00:09:22.411588 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:22.403322 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 00:09:22.411588 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.403330 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 00:09:22.411588 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.403489 2558 server.go:962] "Client rotation is on, will bootstrap in background" Jul 05 00:09:22.411588 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.406224 2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 05 00:09:22.411588 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.407737 2558 server.go:1019] "Starting client certificate rotation" Jul 05 00:09:22.411588 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.407840 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 00:09:22.411588 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.407873 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 00:09:22.411588 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.411157 2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 00:09:22.416221 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.416198 2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 00:09:22.426074 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.426053 2558 log.go:25] "Validated CRI v1 runtime API" Jul 05 00:09:22.428094 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.428078 2558 log.go:25] "Validated CRI v1 image API" Jul 05 00:09:22.428818 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.428802 2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 05 00:09:22.431567 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.431549 2558 fs.go:135] Filesystem UUIDs: map[6581c824-85e6-48ec-8f4d-6e8ca01349fe:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a4c964ea-f7bb-4697-98d4-6936f8afcb47:/dev/nvme0n1p4] Jul 05 00:09:22.431625 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.431566 2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 05 00:09:22.436408 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.436301 2558 manager.go:216] Machine: {Timestamp:2026-07-05 00:09:22.435234562 +0000 UTC m=+0.371599932 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104214 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2640ede6cbf8d69749a336ca255c64 SystemUUID:ec2640ed-e6cb-f8d6-9749-a336ca255c64 BootID:1c36ad4e-f607-4df4-b302-947e0d2cd2ff Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5d:47:60:45:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5d:47:60:45:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:e3:e2:64:e9:df Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 05 00:09:22.436408 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.436402 2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 05 00:09:22.436543 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.436477 2558 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 05 00:09:22.437070 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.437054 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 00:09:22.437497 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.437466 2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 05 00:09:22.437656 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.437503 2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-0.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 05 00:09:22.438734 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.438720 2558 topology_manager.go:138] "Creating topology manager with none policy" Jul 05 00:09:22.438795 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.438739 2558 container_manager_linux.go:309] "Creating device plugin manager" Jul 05 00:09:22.438795 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.438750 2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 00:09:22.439378 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.439366 2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 00:09:22.440917 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.440892 2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 05 00:09:22.443751 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.443736 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 05 00:09:22.443885 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.443873 2558 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 05 00:09:22.443966 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.443928 2558 kubelet.go:486] "Attempting to sync node with API server" Jul 05 00:09:22.443966 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.443944 2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 05 00:09:22.443966 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.443963 2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 05 00:09:22.444094 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.443975 2558 kubelet.go:398] "Adding apiserver pod source" Jul 05 00:09:22.444094 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.443988 2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 05 00:09:22.444354 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.444339 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 00:09:22.444416 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.444363 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 00:09:22.446740 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.446725 2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 05 00:09:22.448699 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.448686 2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 05 00:09:22.448737 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.448722 2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 05 00:09:22.449995 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.449984 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 05 00:09:22.450038 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.449998 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 05 00:09:22.450038 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.450004 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 05 00:09:22.450038 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.450009 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 05 00:09:22.450038 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.450018 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 05 00:09:22.450038 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.450024 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 05 00:09:22.450038 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.450035 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 05 00:09:22.450194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.450041 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 05 00:09:22.450194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.450049 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 05 00:09:22.450194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.450055 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 05 00:09:22.450194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.450069 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 05 00:09:22.450194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.450075 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 05 00:09:22.450804 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.450793 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 05 00:09:22.450834 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.450805 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 05 00:09:22.454043 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.454023 2558 server.go:1268] "Started kubelet" Jul 05 00:09:22.454140 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.454081 2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 05 00:09:22.454230 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.454165 2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 05 00:09:22.454329 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.454273 2558 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 05 00:09:22.455001 ip-10-0-130-0 systemd[1]: Started Kubernetes Kubelet. Jul 05 00:09:22.457843 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.457824 2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 05 00:09:22.458429 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.458400 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qmxj4" Jul 05 00:09:22.459920 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.459892 2558 server.go:310] "Adding debug handlers to kubelet server" Jul 05 00:09:22.461796 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.461758 2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-0.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 05 00:09:22.461884 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.461856 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 05 00:09:22.462015 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.461895 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-0.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 05 00:09:22.462336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.462318 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qmxj4" Jul 05 00:09:22.462421 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.462368 2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 05 00:09:22.469443 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.469426 2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 05 00:09:22.469443 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.469435 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 05 00:09:22.469571 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.469520 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:22.470122 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.470099 2558 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 05 00:09:22.470209 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.470170 2558 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 05 00:09:22.470209 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.470099 2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 05 00:09:22.470339 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.470233 2558 factory.go:55] Registering systemd factory Jul 05 00:09:22.470339 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.470246 2558 factory.go:223] Registration of the systemd container factory successfully Jul 05 00:09:22.470339 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.470272 2558 reconstruct.go:97] "Volume reconstruction finished" Jul 05 00:09:22.470339 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.470279 2558 reconciler.go:29] "Reconciler: start to sync state" Jul 05 00:09:22.470681 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.470667 2558 factory.go:153] Registering CRI-O factory Jul 05 00:09:22.470729 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.470682 2558 factory.go:223] Registration of the crio container factory successfully Jul 05 00:09:22.470774 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.470735 2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 05 00:09:22.470774 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.470759 2558 factory.go:103] Registering Raw factory Jul 05 00:09:22.470774 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.470774 2558 manager.go:1196] Started watching for new ooms in manager Jul 05 00:09:22.471225 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.471210 2558 manager.go:349] Starting recovery of all containers Jul 05 00:09:22.472119 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.472099 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 00:09:22.475793 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.475770 2558 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-0.ec2.internal\" not found" node="ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.481394 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.481293 2558 manager.go:354] Recovery completed Jul 05 00:09:22.485562 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.485549 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 00:09:22.487476 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.487457 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasSufficientMemory" Jul 05 00:09:22.487552 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.487493 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 00:09:22.487552 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.487506 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasSufficientPID" Jul 05 00:09:22.488000 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.487988 2558 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 05 00:09:22.488047 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.487999 2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 05 00:09:22.488047 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.488015 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 05 00:09:22.488047 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.488035 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 00:09:22.488713 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.488700 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasSufficientMemory" Jul 05 00:09:22.488753 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.488732 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 00:09:22.488753 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.488744 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasSufficientPID" Jul 05 00:09:22.489048 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.489037 2558 policy_none.go:49] "None policy: Start" Jul 05 00:09:22.489080 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.489053 2558 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 05 00:09:22.489080 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.489063 2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 05 00:09:22.490082 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.490071 2558 policy_none.go:47] "Start" Jul 05 00:09:22.527453 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.527439 2558 manager.go:337] "Starting Device Plugin manager" Jul 05 00:09:22.527546 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.527470 2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 05 00:09:22.527546 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.527478 2558 server.go:87] "Starting device plugin registration server" Jul 05 00:09:22.527691 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.527680 2558 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 05 00:09:22.527737 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.527693 2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 05 00:09:22.527792 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.527775 2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 05 00:09:22.527884 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.527849 2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 05 00:09:22.527884 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.527864 2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 05 00:09:22.528414 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.528394 2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 05 00:09:22.528493 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.528450 2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:22.616232 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.616201 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 05 00:09:22.617481 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.617468 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 05 00:09:22.617529 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.617500 2558 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 05 00:09:22.617529 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.617528 2558 kubelet.go:2448] "Starting kubelet main sync loop" Jul 05 00:09:22.617590 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.617556 2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 05 00:09:22.620223 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.620203 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 00:09:22.628448 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.628430 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 00:09:22.629350 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.629332 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasSufficientMemory" Jul 05 00:09:22.629431 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.629364 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 00:09:22.629431 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.629376 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasSufficientPID" Jul 05 00:09:22.629431 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.629396 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.638742 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.638697 2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.638742 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.638717 2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-0.ec2.internal\": node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:22.651090 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.651067 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:22.718451 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.718415 2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-0.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal"] Jul 05 00:09:22.718565 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.718489 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 00:09:22.719889 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.719871 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasSufficientMemory" Jul 05 00:09:22.720004 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.719897 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 00:09:22.720004 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.719927 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasSufficientPID" Jul 05 00:09:22.721251 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.721239 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 00:09:22.721825 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.721811 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.721881 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.721844 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 00:09:22.721974 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.721961 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasSufficientMemory" Jul 05 00:09:22.722015 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.721986 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 00:09:22.722015 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.721999 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasSufficientPID" Jul 05 00:09:22.722465 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.722450 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasSufficientMemory" Jul 05 00:09:22.722527 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.722478 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 00:09:22.722527 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.722489 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasSufficientPID" Jul 05 00:09:22.723156 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.723139 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.723256 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.723173 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 00:09:22.724456 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.724441 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasSufficientMemory" Jul 05 00:09:22.724526 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.724466 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 00:09:22.724526 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.724475 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeHasSufficientPID" Jul 05 00:09:22.746537 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.746513 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-0.ec2.internal\" not found" node="ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.750640 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.750626 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-0.ec2.internal\" not found" node="ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.751627 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.751609 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:22.772276 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.772259 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb4e5006e9150c09aadf1dc6b5f68292-config\") pod \"kube-apiserver-proxy-ip-10-0-130-0.ec2.internal\" (UID: \"fb4e5006e9150c09aadf1dc6b5f68292\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.772363 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.772297 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d76e3a0c4ee450c5a2e97e058e2dfe70-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal\" (UID: \"d76e3a0c4ee450c5a2e97e058e2dfe70\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.772363 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.772323 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d76e3a0c4ee450c5a2e97e058e2dfe70-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal\" (UID: \"d76e3a0c4ee450c5a2e97e058e2dfe70\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.851898 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.851874 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:22.873305 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.873278 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb4e5006e9150c09aadf1dc6b5f68292-config\") pod \"kube-apiserver-proxy-ip-10-0-130-0.ec2.internal\" (UID: \"fb4e5006e9150c09aadf1dc6b5f68292\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.873305 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.873293 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb4e5006e9150c09aadf1dc6b5f68292-config\") pod \"kube-apiserver-proxy-ip-10-0-130-0.ec2.internal\" (UID: \"fb4e5006e9150c09aadf1dc6b5f68292\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.873438 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.873317 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d76e3a0c4ee450c5a2e97e058e2dfe70-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal\" (UID: \"d76e3a0c4ee450c5a2e97e058e2dfe70\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.873438 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.873335 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d76e3a0c4ee450c5a2e97e058e2dfe70-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal\" (UID: \"d76e3a0c4ee450c5a2e97e058e2dfe70\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.873438 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.873356 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d76e3a0c4ee450c5a2e97e058e2dfe70-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal\" (UID: \"d76e3a0c4ee450c5a2e97e058e2dfe70\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.873438 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:22.873369 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d76e3a0c4ee450c5a2e97e058e2dfe70-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal\" (UID: \"d76e3a0c4ee450c5a2e97e058e2dfe70\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal" Jul 05 00:09:22.952770 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:22.952696 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:23.049271 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:23.049238 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-0.ec2.internal" Jul 05 00:09:23.053209 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:23.053152 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:23.054232 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:23.054217 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal" Jul 05 00:09:23.153932 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:23.153876 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:23.255039 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:23.254966 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:23.309089 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:23.309058 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 00:09:23.355790 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:23.355755 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:23.407597 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:23.407574 2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 05 00:09:23.408214 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:23.407710 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 05 00:09:23.408214 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:23.407751 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 05 00:09:23.408214 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:23.407756 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 05 00:09:23.456914 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:23.456876 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:23.468056 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:23.468024 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-04 00:04:22 +0000 UTC" deadline="2028-03-06 20:02:33.1225746 +0000 UTC" Jul 05 00:09:23.468056 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:23.468056 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14659h53m9.654524917s" Jul 05 00:09:23.558100 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:23.558076 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:23.652489 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:23.652450 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb4e5006e9150c09aadf1dc6b5f68292.slice/crio-6388b475c9bd40f1ac6f0da05e14f481978119a340ac39286a24e76f687ab08f WatchSource:0}: Error finding container 6388b475c9bd40f1ac6f0da05e14f481978119a340ac39286a24e76f687ab08f: Status 404 returned error can't find the container with id 6388b475c9bd40f1ac6f0da05e14f481978119a340ac39286a24e76f687ab08f Jul 05 00:09:23.653625 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:23.653600 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd76e3a0c4ee450c5a2e97e058e2dfe70.slice/crio-a69bda06496a10ac0edcecdadda2cc0cdce1b758645f6f6437d44ec6a5500ed5 WatchSource:0}: Error finding container a69bda06496a10ac0edcecdadda2cc0cdce1b758645f6f6437d44ec6a5500ed5: Status 404 returned error can't find the container with id a69bda06496a10ac0edcecdadda2cc0cdce1b758645f6f6437d44ec6a5500ed5 Jul 05 00:09:23.658620 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:23.658593 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:23.660468 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:23.660443 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 00:09:23.761914 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:23.761852 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:23.864022 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:23.863946 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:23.965148 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:23.965118 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-0.ec2.internal\" not found" Jul 05 00:09:23.978894 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:23.978865 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 00:09:24.070568 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.070540 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-0.ec2.internal" Jul 05 00:09:24.081059 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.081031 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 00:09:24.082276 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.082256 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal" Jul 05 00:09:24.093698 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.093677 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 00:09:24.427271 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.427242 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 00:09:24.444645 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.444622 2558 apiserver.go:52] "Watching apiserver" Jul 05 00:09:24.456501 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.456475 2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 05 00:09:24.457003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.456971 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-2g6qj","openshift-dns/node-resolver-9595p","openshift-image-registry/node-ca-8s5zw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal","openshift-multus/multus-additional-cni-plugins-k8wn9","openshift-multus/multus-mxnwd","openshift-ovn-kubernetes/ovnkube-node-56fqd","kube-system/global-pull-secret-syncer-qb4z2","kube-system/konnectivity-agent-qlbnt","kube-system/kube-apiserver-proxy-ip-10-0-130-0.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq","openshift-multus/network-metrics-daemon-td7ll","openshift-network-diagnostics/network-check-target-n62q6","openshift-network-operator/iptables-alerter-qd79j"] Jul 05 00:09:24.459019 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.458397 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.459787 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.459765 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:24.459901 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:24.459847 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:24.465115 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.462059 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:24.465115 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:24.462152 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:24.465115 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.462977 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 05 00:09:24.465115 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.463272 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 05 00:09:24.465115 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.463372 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7l7bt\"" Jul 05 00:09:24.465952 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.465925 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9595p" Jul 05 00:09:24.468442 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.468420 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8s5zw" Jul 05 00:09:24.469345 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.469326 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 05 00:09:24.469462 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.469326 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mlzqh\"" Jul 05 00:09:24.469693 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.469516 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 05 00:09:24.470653 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.470633 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 05 00:09:24.471117 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.471096 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z47bv\"" Jul 05 00:09:24.471445 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.471425 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 05 00:09:24.471763 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.471737 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 05 00:09:24.472075 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.472058 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 05 00:09:24.472160 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.472096 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.473253 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.473232 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.474377 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.474355 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 05 00:09:24.474816 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.474795 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 05 00:09:24.474816 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.474810 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 05 00:09:24.475100 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.475079 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m9nfv\"" Jul 05 00:09:24.475248 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.475093 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 05 00:09:24.475575 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.475556 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 05 00:09:24.475835 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.475585 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.476548 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.476371 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 05 00:09:24.476548 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.476505 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c7flq\"" Jul 05 00:09:24.477521 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.477290 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:24.477521 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:24.477373 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:24.478626 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.478606 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 05 00:09:24.478626 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.478618 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 05 00:09:24.479474 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.479099 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qlbnt" Jul 05 00:09:24.480436 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.480410 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 05 00:09:24.480539 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.480521 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 05 00:09:24.480659 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.480640 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 05 00:09:24.480724 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.480709 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 05 00:09:24.480772 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.480731 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zkpmw\"" Jul 05 00:09:24.481856 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.481835 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 05 00:09:24.482166 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.482146 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qd79j" Jul 05 00:09:24.482821 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.482803 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ht57q\"" Jul 05 00:09:24.482890 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.482874 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-host\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.482932 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.482887 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 05 00:09:24.482932 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.482903 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxrgq\" (UniqueName: \"kubernetes.io/projected/0201eff2-4a8d-4281-954f-137b7b3a8a4d-kube-api-access-jxrgq\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.483057 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.482944 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/215cde65-c249-45c8-b941-488c4ab6b568-tmp-dir\") pod \"node-resolver-9595p\" (UID: \"215cde65-c249-45c8-b941-488c4ab6b568\") " pod="openshift-dns/node-resolver-9595p" Jul 05 00:09:24.483057 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483035 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ns6t\" (UniqueName: \"kubernetes.io/projected/f2fde0b3-a812-4baf-9da4-719e7b227fd9-kube-api-access-8ns6t\") pod \"node-ca-8s5zw\" (UID: \"f2fde0b3-a812-4baf-9da4-719e7b227fd9\") " pod="openshift-image-registry/node-ca-8s5zw" Jul 05 00:09:24.483170 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483076 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/039c4612-e8d2-4bf8-994f-37600257cc9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.483170 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483097 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k6dw\" (UniqueName: \"kubernetes.io/projected/215cde65-c249-45c8-b941-488c4ab6b568-kube-api-access-8k6dw\") pod \"node-resolver-9595p\" (UID: \"215cde65-c249-45c8-b941-488c4ab6b568\") " pod="openshift-dns/node-resolver-9595p" Jul 05 00:09:24.483170 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483128 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2fde0b3-a812-4baf-9da4-719e7b227fd9-host\") pod \"node-ca-8s5zw\" (UID: \"f2fde0b3-a812-4baf-9da4-719e7b227fd9\") " pod="openshift-image-registry/node-ca-8s5zw" Jul 05 00:09:24.483170 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483148 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f2fde0b3-a812-4baf-9da4-719e7b227fd9-serviceca\") pod \"node-ca-8s5zw\" (UID: \"f2fde0b3-a812-4baf-9da4-719e7b227fd9\") " pod="openshift-image-registry/node-ca-8s5zw" Jul 05 00:09:24.483371 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483194 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/039c4612-e8d2-4bf8-994f-37600257cc9b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.483371 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483238 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-sysconfig\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.483371 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483256 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-kubernetes\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.483371 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483290 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.483579 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483412 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 00:09:24.483579 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483276 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-systemd\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.483579 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483534 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m52qq\" (UniqueName: \"kubernetes.io/projected/a8e8f787-142b-4f26-9a75-9cc3b6976f35-kube-api-access-m52qq\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:24.483579 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483561 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-var-lib-kubelet\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.483773 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483593 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-sysctl-conf\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.483773 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483616 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-sys\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.483773 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483663 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-run\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.483773 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483695 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-tuned\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.483773 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483758 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/039c4612-e8d2-4bf8-994f-37600257cc9b-cni-binary-copy\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.484033 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483814 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-sysctl-d\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.484033 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483843 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtzvv\" (UniqueName: \"kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv\") pod \"network-check-target-n62q6\" (UID: \"00006af4-11b4-46e0-afdd-71f770964dc5\") " pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:24.484033 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483876 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/039c4612-e8d2-4bf8-994f-37600257cc9b-cnibin\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.484033 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483896 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/039c4612-e8d2-4bf8-994f-37600257cc9b-os-release\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.484033 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483938 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:24.484033 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483964 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/039c4612-e8d2-4bf8-994f-37600257cc9b-system-cni-dir\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.484033 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483982 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/039c4612-e8d2-4bf8-994f-37600257cc9b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.484033 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.483998 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-modprobe-d\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.484033 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.484013 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-lib-modules\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.484534 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.484055 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0201eff2-4a8d-4281-954f-137b7b3a8a4d-tmp\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.484534 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.484073 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/215cde65-c249-45c8-b941-488c4ab6b568-hosts-file\") pod \"node-resolver-9595p\" (UID: \"215cde65-c249-45c8-b941-488c4ab6b568\") " pod="openshift-dns/node-resolver-9595p" Jul 05 00:09:24.484534 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.484091 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxhlr\" (UniqueName: \"kubernetes.io/projected/039c4612-e8d2-4bf8-994f-37600257cc9b-kube-api-access-lxhlr\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.484823 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.484803 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 05 00:09:24.485215 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.485190 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 05 00:09:24.485422 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.485407 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-czjkq\"" Jul 05 00:09:24.486144 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.485901 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 05 00:09:24.487329 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.486371 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-r6c8s\"" Jul 05 00:09:24.487329 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.486609 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 05 00:09:24.487329 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.486938 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 05 00:09:24.487329 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.487191 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 05 00:09:24.503214 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.503188 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s68wk" Jul 05 00:09:24.509532 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.509511 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s68wk" Jul 05 00:09:24.524820 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.521957 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 00:09:24.571633 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.571610 2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 05 00:09:24.584842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584556 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2fde0b3-a812-4baf-9da4-719e7b227fd9-host\") pod \"node-ca-8s5zw\" (UID: \"f2fde0b3-a812-4baf-9da4-719e7b227fd9\") " pod="openshift-image-registry/node-ca-8s5zw" Jul 05 00:09:24.584842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584598 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-sys-fs\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.584842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584625 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-etc-openvswitch\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.584842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584635 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2fde0b3-a812-4baf-9da4-719e7b227fd9-host\") pod \"node-ca-8s5zw\" (UID: \"f2fde0b3-a812-4baf-9da4-719e7b227fd9\") " pod="openshift-image-registry/node-ca-8s5zw" Jul 05 00:09:24.584842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584651 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a828f18f-4034-4e6f-963f-04e391abbe14-env-overrides\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.584842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584674 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a45b95a1-e42e-406b-9df7-096497d8369e-iptables-alerter-script\") pod \"iptables-alerter-qd79j\" (UID: \"a45b95a1-e42e-406b-9df7-096497d8369e\") " pod="openshift-network-operator/iptables-alerter-qd79j" Jul 05 00:09:24.584842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584702 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-kubernetes\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.584842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584723 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m52qq\" (UniqueName: \"kubernetes.io/projected/a8e8f787-142b-4f26-9a75-9cc3b6976f35-kube-api-access-m52qq\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:24.584842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584739 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-registration-dir\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.584842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584761 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7ml6\" (UniqueName: \"kubernetes.io/projected/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-kube-api-access-b7ml6\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.584842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584798 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-multus-conf-dir\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584888 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-run-netns\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584928 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-kubernetes\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584951 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-run-openvswitch\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.584987 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-var-lib-kubelet\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585021 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-sysctl-conf\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585052 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-etc-selinux\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585113 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-var-lib-kubelet\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585145 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-multus-socket-dir-parent\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585177 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-sysctl-conf\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585182 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-var-lib-kubelet\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585226 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-run-systemd\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585246 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-run-ovn\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585262 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-node-log\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585281 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-run\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585303 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-tuned\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585327 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-system-cni-dir\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.585551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585352 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-var-lib-cni-bin\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585367 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-run\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585372 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a828f18f-4034-4e6f-963f-04e391abbe14-ovnkube-config\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585406 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw662\" (UniqueName: \"kubernetes.io/projected/a828f18f-4034-4e6f-963f-04e391abbe14-kube-api-access-sw662\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585436 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-sysctl-d\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585461 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtzvv\" (UniqueName: \"kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv\") pod \"network-check-target-n62q6\" (UID: \"00006af4-11b4-46e0-afdd-71f770964dc5\") " pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585513 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/039c4612-e8d2-4bf8-994f-37600257cc9b-os-release\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585542 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-cni-binary-copy\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585563 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-hostroot\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585586 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a45b95a1-e42e-406b-9df7-096497d8369e-host-slash\") pod \"iptables-alerter-qd79j\" (UID: \"a45b95a1-e42e-406b-9df7-096497d8369e\") " pod="openshift-network-operator/iptables-alerter-qd79j" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585614 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/039c4612-e8d2-4bf8-994f-37600257cc9b-system-cni-dir\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585627 2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585638 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fa9b29e8-905f-4113-9dd6-cde4021610d8-konnectivity-ca\") pod \"konnectivity-agent-qlbnt\" (UID: \"fa9b29e8-905f-4113-9dd6-cde4021610d8\") " pod="kube-system/konnectivity-agent-qlbnt" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585664 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-run-netns\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585688 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-run-multus-certs\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585714 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-kubelet\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585739 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-lib-modules\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.586463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585764 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/215cde65-c249-45c8-b941-488c4ab6b568-hosts-file\") pod \"node-resolver-9595p\" (UID: \"215cde65-c249-45c8-b941-488c4ab6b568\") " pod="openshift-dns/node-resolver-9595p" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585796 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxhlr\" (UniqueName: \"kubernetes.io/projected/039c4612-e8d2-4bf8-994f-37600257cc9b-kube-api-access-lxhlr\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585822 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-etc-kubernetes\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585850 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-host\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585876 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxrgq\" (UniqueName: \"kubernetes.io/projected/0201eff2-4a8d-4281-954f-137b7b3a8a4d-kube-api-access-jxrgq\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.585904 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/215cde65-c249-45c8-b941-488c4ab6b568-tmp-dir\") pod \"node-resolver-9595p\" (UID: \"215cde65-c249-45c8-b941-488c4ab6b568\") " pod="openshift-dns/node-resolver-9595p" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586018 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/039c4612-e8d2-4bf8-994f-37600257cc9b-system-cni-dir\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586068 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/215cde65-c249-45c8-b941-488c4ab6b568-hosts-file\") pod \"node-resolver-9595p\" (UID: \"215cde65-c249-45c8-b941-488c4ab6b568\") " pod="openshift-dns/node-resolver-9595p" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586182 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-lib-modules\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586224 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/039c4612-e8d2-4bf8-994f-37600257cc9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586254 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/215cde65-c249-45c8-b941-488c4ab6b568-tmp-dir\") pod \"node-resolver-9595p\" (UID: \"215cde65-c249-45c8-b941-488c4ab6b568\") " pod="openshift-dns/node-resolver-9595p" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586307 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-host\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586339 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-os-release\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586356 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-sysctl-d\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586365 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-multus-daemon-config\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586425 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/039c4612-e8d2-4bf8-994f-37600257cc9b-os-release\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586447 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/039c4612-e8d2-4bf8-994f-37600257cc9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.587336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586494 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k6dw\" (UniqueName: \"kubernetes.io/projected/215cde65-c249-45c8-b941-488c4ab6b568-kube-api-access-8k6dw\") pod \"node-resolver-9595p\" (UID: \"215cde65-c249-45c8-b941-488c4ab6b568\") " pod="openshift-dns/node-resolver-9595p" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586578 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f2fde0b3-a812-4baf-9da4-719e7b227fd9-serviceca\") pod \"node-ca-8s5zw\" (UID: \"f2fde0b3-a812-4baf-9da4-719e7b227fd9\") " pod="openshift-image-registry/node-ca-8s5zw" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586606 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/039c4612-e8d2-4bf8-994f-37600257cc9b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586628 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ef46745-1efb-44fe-8511-59c406acd812-dbus\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586657 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2nqt\" (UniqueName: \"kubernetes.io/projected/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-kube-api-access-r2nqt\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586699 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-sysconfig\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586721 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-systemd\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586736 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586753 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-device-dir\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586784 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-cni-bin\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586801 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-sys\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586824 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-log-socket\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586849 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-run-ovn-kubernetes\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586867 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-cni-netd\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586883 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/039c4612-e8d2-4bf8-994f-37600257cc9b-cni-binary-copy\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.586910 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-socket-dir\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587056 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-multus-cni-dir\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.588217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587074 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-run-k8s-cni-cncf-io\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587097 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4f9l\" (UniqueName: \"kubernetes.io/projected/a45b95a1-e42e-406b-9df7-096497d8369e-kube-api-access-c4f9l\") pod \"iptables-alerter-qd79j\" (UID: \"a45b95a1-e42e-406b-9df7-096497d8369e\") " pod="openshift-network-operator/iptables-alerter-qd79j" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587139 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/039c4612-e8d2-4bf8-994f-37600257cc9b-cnibin\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587166 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ef46745-1efb-44fe-8511-59c406acd812-kubelet-config\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587182 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fa9b29e8-905f-4113-9dd6-cde4021610d8-agent-certs\") pod \"konnectivity-agent-qlbnt\" (UID: \"fa9b29e8-905f-4113-9dd6-cde4021610d8\") " pod="kube-system/konnectivity-agent-qlbnt" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587201 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-systemd-units\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587228 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-var-lib-openvswitch\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587247 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f2fde0b3-a812-4baf-9da4-719e7b227fd9-serviceca\") pod \"node-ca-8s5zw\" (UID: \"f2fde0b3-a812-4baf-9da4-719e7b227fd9\") " pod="openshift-image-registry/node-ca-8s5zw" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587247 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587304 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-sysconfig\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587323 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/039c4612-e8d2-4bf8-994f-37600257cc9b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587352 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/039c4612-e8d2-4bf8-994f-37600257cc9b-cnibin\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587367 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a828f18f-4034-4e6f-963f-04e391abbe14-ovn-node-metrics-cert\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587402 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587430 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/039c4612-e8d2-4bf8-994f-37600257cc9b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587443 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-sys\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587455 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-cnibin\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.588826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587481 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-var-lib-cni-multus\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.589459 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587490 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-systemd\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.589459 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587506 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-slash\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.589459 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587565 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a828f18f-4034-4e6f-963f-04e391abbe14-ovnkube-script-lib\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.589459 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587594 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-modprobe-d\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.589459 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:24.587618 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 00:09:24.589459 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:24.587697 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs podName:a8e8f787-142b-4f26-9a75-9cc3b6976f35 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:25.087662365 +0000 UTC m=+3.024027744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs") pod "network-metrics-daemon-td7ll" (UID: "a8e8f787-142b-4f26-9a75-9cc3b6976f35") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 00:09:24.589459 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587709 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/039c4612-e8d2-4bf8-994f-37600257cc9b-cni-binary-copy\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.589459 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587620 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0201eff2-4a8d-4281-954f-137b7b3a8a4d-tmp\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.589459 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587779 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ns6t\" (UniqueName: \"kubernetes.io/projected/f2fde0b3-a812-4baf-9da4-719e7b227fd9-kube-api-access-8ns6t\") pod \"node-ca-8s5zw\" (UID: \"f2fde0b3-a812-4baf-9da4-719e7b227fd9\") " pod="openshift-image-registry/node-ca-8s5zw" Jul 05 00:09:24.589459 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.587809 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:24.589459 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.588026 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-modprobe-d\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.589459 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.588509 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/039c4612-e8d2-4bf8-994f-37600257cc9b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.589459 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.589164 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0201eff2-4a8d-4281-954f-137b7b3a8a4d-etc-tuned\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.590353 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.590329 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0201eff2-4a8d-4281-954f-137b7b3a8a4d-tmp\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.594071 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:24.593437 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 00:09:24.594071 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:24.593465 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 00:09:24.594071 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:24.593481 2558 projected.go:196] Error preparing data for projected volume kube-api-access-dtzvv for pod openshift-network-diagnostics/network-check-target-n62q6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 00:09:24.594071 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:24.593593 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv podName:00006af4-11b4-46e0-afdd-71f770964dc5 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:25.093576252 +0000 UTC m=+3.029941633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dtzvv" (UniqueName: "kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv") pod "network-check-target-n62q6" (UID: "00006af4-11b4-46e0-afdd-71f770964dc5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 00:09:24.594071 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.594013 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m52qq\" (UniqueName: \"kubernetes.io/projected/a8e8f787-142b-4f26-9a75-9cc3b6976f35-kube-api-access-m52qq\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:24.595124 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.595099 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxhlr\" (UniqueName: \"kubernetes.io/projected/039c4612-e8d2-4bf8-994f-37600257cc9b-kube-api-access-lxhlr\") pod \"multus-additional-cni-plugins-k8wn9\" (UID: \"039c4612-e8d2-4bf8-994f-37600257cc9b\") " pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.602039 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.602016 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ns6t\" (UniqueName: \"kubernetes.io/projected/f2fde0b3-a812-4baf-9da4-719e7b227fd9-kube-api-access-8ns6t\") pod \"node-ca-8s5zw\" (UID: \"f2fde0b3-a812-4baf-9da4-719e7b227fd9\") " pod="openshift-image-registry/node-ca-8s5zw" Jul 05 00:09:24.602622 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.602600 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxrgq\" (UniqueName: \"kubernetes.io/projected/0201eff2-4a8d-4281-954f-137b7b3a8a4d-kube-api-access-jxrgq\") pod \"tuned-2g6qj\" (UID: \"0201eff2-4a8d-4281-954f-137b7b3a8a4d\") " pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.604254 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.604233 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k6dw\" (UniqueName: \"kubernetes.io/projected/215cde65-c249-45c8-b941-488c4ab6b568-kube-api-access-8k6dw\") pod \"node-resolver-9595p\" (UID: \"215cde65-c249-45c8-b941-488c4ab6b568\") " pod="openshift-dns/node-resolver-9595p" Jul 05 00:09:24.621846 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.621801 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-0.ec2.internal" event={"ID":"fb4e5006e9150c09aadf1dc6b5f68292","Type":"ContainerStarted","Data":"6388b475c9bd40f1ac6f0da05e14f481978119a340ac39286a24e76f687ab08f"} Jul 05 00:09:24.623101 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.623075 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal" event={"ID":"d76e3a0c4ee450c5a2e97e058e2dfe70","Type":"ContainerStarted","Data":"a69bda06496a10ac0edcecdadda2cc0cdce1b758645f6f6437d44ec6a5500ed5"} Jul 05 00:09:24.689375 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689258 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-socket-dir\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.689375 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689356 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-multus-cni-dir\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.689576 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689399 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-run-k8s-cni-cncf-io\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.689576 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689437 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4f9l\" (UniqueName: \"kubernetes.io/projected/a45b95a1-e42e-406b-9df7-096497d8369e-kube-api-access-c4f9l\") pod \"iptables-alerter-qd79j\" (UID: \"a45b95a1-e42e-406b-9df7-096497d8369e\") " pod="openshift-network-operator/iptables-alerter-qd79j" Jul 05 00:09:24.689576 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689477 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ef46745-1efb-44fe-8511-59c406acd812-kubelet-config\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:24.689576 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689519 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fa9b29e8-905f-4113-9dd6-cde4021610d8-agent-certs\") pod \"konnectivity-agent-qlbnt\" (UID: \"fa9b29e8-905f-4113-9dd6-cde4021610d8\") " pod="kube-system/konnectivity-agent-qlbnt" Jul 05 00:09:24.689576 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689548 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-systemd-units\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.689737 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689583 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-var-lib-openvswitch\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.689737 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689619 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.689737 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689656 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a828f18f-4034-4e6f-963f-04e391abbe14-ovn-node-metrics-cert\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.689737 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689708 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-cnibin\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.689737 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689730 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-var-lib-cni-multus\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.689885 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689748 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-slash\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.689885 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689767 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a828f18f-4034-4e6f-963f-04e391abbe14-ovnkube-script-lib\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.689885 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689792 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:24.689885 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689813 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-sys-fs\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.689885 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689833 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-etc-openvswitch\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.689885 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689854 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a828f18f-4034-4e6f-963f-04e391abbe14-env-overrides\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.689885 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689872 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a45b95a1-e42e-406b-9df7-096497d8369e-iptables-alerter-script\") pod \"iptables-alerter-qd79j\" (UID: \"a45b95a1-e42e-406b-9df7-096497d8369e\") " pod="openshift-network-operator/iptables-alerter-qd79j" Jul 05 00:09:24.690151 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689896 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-registration-dir\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.690151 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689918 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7ml6\" (UniqueName: \"kubernetes.io/projected/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-kube-api-access-b7ml6\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.690151 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689939 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-multus-conf-dir\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.690151 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689956 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-run-netns\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.690151 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.689998 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-run-openvswitch\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.690151 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.690034 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-etc-selinux\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.690151 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.690046 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-socket-dir\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.690151 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.690069 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-multus-socket-dir-parent\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.690151 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.690104 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-var-lib-kubelet\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.690151 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.690124 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-slash\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.690151 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.690133 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-run-systemd\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.690522 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.690189 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-run-systemd\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.690522 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.690197 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-multus-cni-dir\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.690522 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.690234 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-run-ovn\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.690522 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:24.690300 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 00:09:24.690522 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.690300 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-run-k8s-cni-cncf-io\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.690522 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:24.690352 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret podName:7ef46745-1efb-44fe-8511-59c406acd812 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:25.190337502 +0000 UTC m=+3.126702860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret") pod "global-pull-secret-syncer-qb4z2" (UID: "7ef46745-1efb-44fe-8511-59c406acd812") : object "kube-system"/"original-pull-secret" not registered Jul 05 00:09:24.690746 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.690552 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-sys-fs\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.690746 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.690597 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-etc-openvswitch\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.690746 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.690640 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ef46745-1efb-44fe-8511-59c406acd812-kubelet-config\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:24.691038 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.691014 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a828f18f-4034-4e6f-963f-04e391abbe14-env-overrides\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.691437 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a45b95a1-e42e-406b-9df7-096497d8369e-iptables-alerter-script\") pod \"iptables-alerter-qd79j\" (UID: \"a45b95a1-e42e-406b-9df7-096497d8369e\") " pod="openshift-network-operator/iptables-alerter-qd79j" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.691536 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-registration-dir\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.691546 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-cnibin\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.691624 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-var-lib-cni-multus\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.691678 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-run-ovn\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.691840 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-multus-conf-dir\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.691876 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-systemd-units\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.691902 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-run-netns\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.691962 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-run-openvswitch\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.691977 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-var-lib-openvswitch\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692054 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-multus-socket-dir-parent\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692056 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-var-lib-kubelet\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692087 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-node-log\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692129 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-system-cni-dir\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692162 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-var-lib-cni-bin\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692194 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a828f18f-4034-4e6f-963f-04e391abbe14-ovnkube-config\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692221 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw662\" (UniqueName: \"kubernetes.io/projected/a828f18f-4034-4e6f-963f-04e391abbe14-kube-api-access-sw662\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.694959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692232 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692284 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a828f18f-4034-4e6f-963f-04e391abbe14-ovnkube-script-lib\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692295 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-cni-binary-copy\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692291 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-node-log\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692331 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-hostroot\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692374 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a45b95a1-e42e-406b-9df7-096497d8369e-host-slash\") pod \"iptables-alerter-qd79j\" (UID: \"a45b95a1-e42e-406b-9df7-096497d8369e\") " pod="openshift-network-operator/iptables-alerter-qd79j" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692381 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-var-lib-cni-bin\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692412 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fa9b29e8-905f-4113-9dd6-cde4021610d8-konnectivity-ca\") pod \"konnectivity-agent-qlbnt\" (UID: \"fa9b29e8-905f-4113-9dd6-cde4021610d8\") " pod="kube-system/konnectivity-agent-qlbnt" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692434 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-hostroot\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692442 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-run-netns\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692475 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-run-multus-certs\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692512 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-kubelet\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692551 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-etc-kubernetes\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692587 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-os-release\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692628 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-multus-daemon-config\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692664 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ef46745-1efb-44fe-8511-59c406acd812-dbus\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692697 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2nqt\" (UniqueName: \"kubernetes.io/projected/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-kube-api-access-r2nqt\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692734 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.696003 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692763 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-device-dir\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692798 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-cni-bin\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692832 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-log-socket\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692866 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-run-ovn-kubernetes\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692866 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a828f18f-4034-4e6f-963f-04e391abbe14-ovnkube-config\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692899 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-cni-netd\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692966 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-os-release\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693045 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-cni-netd\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693231 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-cni-binary-copy\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693287 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693313 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-cni-bin\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693371 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-run-ovn-kubernetes\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693380 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-device-dir\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693418 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-log-socket\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693481 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a45b95a1-e42e-406b-9df7-096497d8369e-host-slash\") pod \"iptables-alerter-qd79j\" (UID: \"a45b95a1-e42e-406b-9df7-096497d8369e\") " pod="openshift-network-operator/iptables-alerter-qd79j" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692341 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-system-cni-dir\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.692194 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-etc-selinux\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693559 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-run-multus-certs\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.696970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693575 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-multus-daemon-config\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.697968 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693609 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-host-run-netns\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.697968 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693660 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a828f18f-4034-4e6f-963f-04e391abbe14-host-kubelet\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.697968 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693714 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-etc-kubernetes\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.697968 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693743 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ef46745-1efb-44fe-8511-59c406acd812-dbus\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:24.697968 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.693885 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fa9b29e8-905f-4113-9dd6-cde4021610d8-konnectivity-ca\") pod \"konnectivity-agent-qlbnt\" (UID: \"fa9b29e8-905f-4113-9dd6-cde4021610d8\") " pod="kube-system/konnectivity-agent-qlbnt" Jul 05 00:09:24.697968 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.695449 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a828f18f-4034-4e6f-963f-04e391abbe14-ovn-node-metrics-cert\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.697968 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.695897 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fa9b29e8-905f-4113-9dd6-cde4021610d8-agent-certs\") pod \"konnectivity-agent-qlbnt\" (UID: \"fa9b29e8-905f-4113-9dd6-cde4021610d8\") " pod="kube-system/konnectivity-agent-qlbnt" Jul 05 00:09:24.700150 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.700120 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4f9l\" (UniqueName: \"kubernetes.io/projected/a45b95a1-e42e-406b-9df7-096497d8369e-kube-api-access-c4f9l\") pod \"iptables-alerter-qd79j\" (UID: \"a45b95a1-e42e-406b-9df7-096497d8369e\") " pod="openshift-network-operator/iptables-alerter-qd79j" Jul 05 00:09:24.701179 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.701156 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7ml6\" (UniqueName: \"kubernetes.io/projected/e97fe81d-cff9-4c26-944b-030bd1d1a1c9-kube-api-access-b7ml6\") pod \"aws-ebs-csi-driver-node-7gbbq\" (UID: \"e97fe81d-cff9-4c26-944b-030bd1d1a1c9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:24.701634 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.701608 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2nqt\" (UniqueName: \"kubernetes.io/projected/eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1-kube-api-access-r2nqt\") pod \"multus-mxnwd\" (UID: \"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1\") " pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.701754 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.701714 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw662\" (UniqueName: \"kubernetes.io/projected/a828f18f-4034-4e6f-963f-04e391abbe14-kube-api-access-sw662\") pod \"ovnkube-node-56fqd\" (UID: \"a828f18f-4034-4e6f-963f-04e391abbe14\") " pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.774778 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.774743 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" Jul 05 00:09:24.785097 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.785072 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9595p" Jul 05 00:09:24.792843 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.792820 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8s5zw" Jul 05 00:09:24.800655 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.800627 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k8wn9" Jul 05 00:09:24.808296 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.808276 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mxnwd" Jul 05 00:09:24.815855 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.815838 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:24.827730 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.827702 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qlbnt" Jul 05 00:09:24.842584 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.842559 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qd79j" Jul 05 00:09:24.847426 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:24.847407 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" Jul 05 00:09:25.096545 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:25.096466 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtzvv\" (UniqueName: \"kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv\") pod \"network-check-target-n62q6\" (UID: \"00006af4-11b4-46e0-afdd-71f770964dc5\") " pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:25.096545 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:25.096525 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:25.096742 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:25.096623 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 00:09:25.096742 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:25.096656 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 00:09:25.096742 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:25.096670 2558 projected.go:196] Error preparing data for projected volume kube-api-access-dtzvv for pod openshift-network-diagnostics/network-check-target-n62q6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 00:09:25.096742 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:25.096625 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 00:09:25.096742 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:25.096731 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv podName:00006af4-11b4-46e0-afdd-71f770964dc5 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:26.096712343 +0000 UTC m=+4.033077719 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dtzvv" (UniqueName: "kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv") pod "network-check-target-n62q6" (UID: "00006af4-11b4-46e0-afdd-71f770964dc5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 00:09:25.096952 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:25.096774 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs podName:a8e8f787-142b-4f26-9a75-9cc3b6976f35 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:26.096759787 +0000 UTC m=+4.033125146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs") pod "network-metrics-daemon-td7ll" (UID: "a8e8f787-142b-4f26-9a75-9cc3b6976f35") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 00:09:25.197800 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:25.197770 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:25.198010 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:25.197957 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 00:09:25.198063 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:25.198026 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret podName:7ef46745-1efb-44fe-8511-59c406acd812 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:26.198004283 +0000 UTC m=+4.134369644 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret") pod "global-pull-secret-syncer-qb4z2" (UID: "7ef46745-1efb-44fe-8511-59c406acd812") : object "kube-system"/"original-pull-secret" not registered Jul 05 00:09:25.512124 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:25.512044 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 00:04:24 +0000 UTC" deadline="2027-12-24 11:01:43.396626521 +0000 UTC" Jul 05 00:09:25.512124 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:25.512118 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12898h52m17.884513831s" Jul 05 00:09:25.690373 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:25.690332 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda828f18f_4034_4e6f_963f_04e391abbe14.slice/crio-cfa9fd6544cdc5171ff1787883d8d27f85a591868ce785f32d3e31c67be1549e WatchSource:0}: Error finding container cfa9fd6544cdc5171ff1787883d8d27f85a591868ce785f32d3e31c67be1549e: Status 404 returned error can't find the container with id cfa9fd6544cdc5171ff1787883d8d27f85a591868ce785f32d3e31c67be1549e Jul 05 00:09:25.692326 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:25.692243 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2fde0b3_a812_4baf_9da4_719e7b227fd9.slice/crio-1f37854e02a59f8d2ea894718933f3aa67045a83d77009109c41e9458775397f WatchSource:0}: Error finding container 1f37854e02a59f8d2ea894718933f3aa67045a83d77009109c41e9458775397f: Status 404 returned error can't find the container with id 1f37854e02a59f8d2ea894718933f3aa67045a83d77009109c41e9458775397f Jul 05 00:09:25.699125 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:25.699072 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod215cde65_c249_45c8_b941_488c4ab6b568.slice/crio-0bfdc45c8e66c1bd05c63d03fff66a6ae9bb512a7b928803284d2166cc52f482 WatchSource:0}: Error finding container 0bfdc45c8e66c1bd05c63d03fff66a6ae9bb512a7b928803284d2166cc52f482: Status 404 returned error can't find the container with id 0bfdc45c8e66c1bd05c63d03fff66a6ae9bb512a7b928803284d2166cc52f482 Jul 05 00:09:25.700370 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:25.700346 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda45b95a1_e42e_406b_9df7_096497d8369e.slice/crio-8eac7097c20dd74afab9b23a34286207b3cfda86858fd4a4aedca8d20ba5f396 WatchSource:0}: Error finding container 8eac7097c20dd74afab9b23a34286207b3cfda86858fd4a4aedca8d20ba5f396: Status 404 returned error can't find the container with id 8eac7097c20dd74afab9b23a34286207b3cfda86858fd4a4aedca8d20ba5f396 Jul 05 00:09:25.702176 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:25.701948 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda5c663_9ac6_4cbf_a3bc_eb37fbe649d1.slice/crio-344575fa406948019163c45e6ed39b3c27276460b6a0d26c4b5f195934b3350e WatchSource:0}: Error finding container 344575fa406948019163c45e6ed39b3c27276460b6a0d26c4b5f195934b3350e: Status 404 returned error can't find the container with id 344575fa406948019163c45e6ed39b3c27276460b6a0d26c4b5f195934b3350e Jul 05 00:09:25.704627 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:25.703661 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9b29e8_905f_4113_9dd6_cde4021610d8.slice/crio-3d6950f5fcf828472a6a27192a2278b7d2f1f1661e80066722d1337ccb7964f7 WatchSource:0}: Error finding container 3d6950f5fcf828472a6a27192a2278b7d2f1f1661e80066722d1337ccb7964f7: Status 404 returned error can't find the container with id 3d6950f5fcf828472a6a27192a2278b7d2f1f1661e80066722d1337ccb7964f7 Jul 05 00:09:25.706199 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:25.706158 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039c4612_e8d2_4bf8_994f_37600257cc9b.slice/crio-13278c27f448c855cca2a2b73ffe3e9c42e2f15f5256ef02b94388b54d857083 WatchSource:0}: Error finding container 13278c27f448c855cca2a2b73ffe3e9c42e2f15f5256ef02b94388b54d857083: Status 404 returned error can't find the container with id 13278c27f448c855cca2a2b73ffe3e9c42e2f15f5256ef02b94388b54d857083 Jul 05 00:09:25.916847 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:25.916819 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 00:09:26.103130 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.103096 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtzvv\" (UniqueName: \"kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv\") pod \"network-check-target-n62q6\" (UID: \"00006af4-11b4-46e0-afdd-71f770964dc5\") " pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:26.103301 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.103152 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:26.103301 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:26.103261 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 00:09:26.103301 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:26.103266 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 00:09:26.103301 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:26.103296 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 00:09:26.103475 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:26.103309 2558 projected.go:196] Error preparing data for projected volume kube-api-access-dtzvv for pod openshift-network-diagnostics/network-check-target-n62q6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 00:09:26.103475 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:26.103321 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs podName:a8e8f787-142b-4f26-9a75-9cc3b6976f35 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:28.103304076 +0000 UTC m=+6.039669437 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs") pod "network-metrics-daemon-td7ll" (UID: "a8e8f787-142b-4f26-9a75-9cc3b6976f35") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 00:09:26.103475 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:26.103360 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv podName:00006af4-11b4-46e0-afdd-71f770964dc5 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:28.103343074 +0000 UTC m=+6.039708455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dtzvv" (UniqueName: "kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv") pod "network-check-target-n62q6" (UID: "00006af4-11b4-46e0-afdd-71f770964dc5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 00:09:26.203817 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.203723 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:26.204016 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:26.203933 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 00:09:26.204016 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:26.204009 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret podName:7ef46745-1efb-44fe-8511-59c406acd812 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:28.203989377 +0000 UTC m=+6.140354735 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret") pod "global-pull-secret-syncer-qb4z2" (UID: "7ef46745-1efb-44fe-8511-59c406acd812") : object "kube-system"/"original-pull-secret" not registered Jul 05 00:09:26.512642 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.512529 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 00:04:24 +0000 UTC" deadline="2027-12-12 22:48:49.84930338 +0000 UTC" Jul 05 00:09:26.512642 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.512580 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12622h39m23.336729234s" Jul 05 00:09:26.623113 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.621305 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:26.623113 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:26.621484 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:26.623113 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.621892 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:26.623113 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:26.622009 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:26.623113 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.622075 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:26.623113 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:26.622154 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:26.652857 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.652822 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qlbnt" event={"ID":"fa9b29e8-905f-4113-9dd6-cde4021610d8","Type":"ContainerStarted","Data":"3d6950f5fcf828472a6a27192a2278b7d2f1f1661e80066722d1337ccb7964f7"} Jul 05 00:09:26.662105 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.662043 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" event={"ID":"e97fe81d-cff9-4c26-944b-030bd1d1a1c9","Type":"ContainerStarted","Data":"ab924cf8bf0b4af4244b9971ee6c7c584599c8a0c825c7f1e40c0566ff52c580"} Jul 05 00:09:26.670403 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.670356 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8s5zw" event={"ID":"f2fde0b3-a812-4baf-9da4-719e7b227fd9","Type":"ContainerStarted","Data":"1f37854e02a59f8d2ea894718933f3aa67045a83d77009109c41e9458775397f"} Jul 05 00:09:26.682225 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.682193 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-0.ec2.internal" event={"ID":"fb4e5006e9150c09aadf1dc6b5f68292","Type":"ContainerStarted","Data":"7e7b9a01245c0b3538ffab4508d57c03f5bac92e01f0595acfd7a7d8f7748c59"} Jul 05 00:09:26.695786 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.694826 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qd79j" event={"ID":"a45b95a1-e42e-406b-9df7-096497d8369e","Type":"ContainerStarted","Data":"8eac7097c20dd74afab9b23a34286207b3cfda86858fd4a4aedca8d20ba5f396"} Jul 05 00:09:26.698000 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.697097 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-0.ec2.internal" podStartSLOduration=2.697079701 podStartE2EDuration="2.697079701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:26.696879837 +0000 UTC m=+4.633245223" watchObservedRunningTime="2026-07-05 00:09:26.697079701 +0000 UTC m=+4.633445086" Jul 05 00:09:26.702177 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.702146 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mxnwd" event={"ID":"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1","Type":"ContainerStarted","Data":"344575fa406948019163c45e6ed39b3c27276460b6a0d26c4b5f195934b3350e"} Jul 05 00:09:26.720713 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.720676 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8wn9" event={"ID":"039c4612-e8d2-4bf8-994f-37600257cc9b","Type":"ContainerStarted","Data":"13278c27f448c855cca2a2b73ffe3e9c42e2f15f5256ef02b94388b54d857083"} Jul 05 00:09:26.727231 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.727105 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9595p" event={"ID":"215cde65-c249-45c8-b941-488c4ab6b568","Type":"ContainerStarted","Data":"0bfdc45c8e66c1bd05c63d03fff66a6ae9bb512a7b928803284d2166cc52f482"} Jul 05 00:09:26.733333 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.733295 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" event={"ID":"a828f18f-4034-4e6f-963f-04e391abbe14","Type":"ContainerStarted","Data":"cfa9fd6544cdc5171ff1787883d8d27f85a591868ce785f32d3e31c67be1549e"} Jul 05 00:09:26.739025 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.738317 2558 generic.go:356] "Generic (PLEG): container finished" podID="d76e3a0c4ee450c5a2e97e058e2dfe70" containerID="2a986cc1c1c9f035f4a026144695afe6dd2beaf927bf4d9d83913cebc8ea344b" exitCode=0 Jul 05 00:09:26.739025 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.738398 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal" event={"ID":"d76e3a0c4ee450c5a2e97e058e2dfe70","Type":"ContainerDied","Data":"2a986cc1c1c9f035f4a026144695afe6dd2beaf927bf4d9d83913cebc8ea344b"} Jul 05 00:09:26.751628 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:26.751599 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" event={"ID":"0201eff2-4a8d-4281-954f-137b7b3a8a4d","Type":"ContainerStarted","Data":"2766c1d961e59a2bf515d1e2e6c311822c464d6c5850ee702964fe78d757818a"} Jul 05 00:09:27.793290 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:27.793196 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal" event={"ID":"d76e3a0c4ee450c5a2e97e058e2dfe70","Type":"ContainerStarted","Data":"28fd0fc7506e4e0145fa49836e4389257b57740250bd4b1d1e86609855368c8c"} Jul 05 00:09:28.121235 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:28.120214 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:28.121235 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:28.120295 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtzvv\" (UniqueName: \"kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv\") pod \"network-check-target-n62q6\" (UID: \"00006af4-11b4-46e0-afdd-71f770964dc5\") " pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:28.121235 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:28.120449 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 00:09:28.121235 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:28.120468 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 00:09:28.121235 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:28.120483 2558 projected.go:196] Error preparing data for projected volume kube-api-access-dtzvv for pod openshift-network-diagnostics/network-check-target-n62q6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 00:09:28.121235 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:28.120542 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv podName:00006af4-11b4-46e0-afdd-71f770964dc5 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:32.120524237 +0000 UTC m=+10.056889617 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dtzvv" (UniqueName: "kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv") pod "network-check-target-n62q6" (UID: "00006af4-11b4-46e0-afdd-71f770964dc5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 00:09:28.121235 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:28.120955 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 00:09:28.121235 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:28.121009 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs podName:a8e8f787-142b-4f26-9a75-9cc3b6976f35 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:32.120992431 +0000 UTC m=+10.057357803 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs") pod "network-metrics-daemon-td7ll" (UID: "a8e8f787-142b-4f26-9a75-9cc3b6976f35") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 00:09:28.221198 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:28.221160 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:28.221373 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:28.221352 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 00:09:28.221432 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:28.221414 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret podName:7ef46745-1efb-44fe-8511-59c406acd812 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:32.221395787 +0000 UTC m=+10.157761149 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret") pod "global-pull-secret-syncer-qb4z2" (UID: "7ef46745-1efb-44fe-8511-59c406acd812") : object "kube-system"/"original-pull-secret" not registered Jul 05 00:09:28.620218 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:28.620184 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:28.620381 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:28.620327 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:28.620711 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:28.620691 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:28.620810 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:28.620790 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:28.620871 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:28.620848 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:28.620954 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:28.620937 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:30.619684 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:30.618855 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:30.619684 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:30.619014 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:30.619684 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:30.619427 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:30.619684 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:30.619533 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:30.620611 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:30.620423 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:30.620611 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:30.620513 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:32.154268 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:32.153554 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtzvv\" (UniqueName: \"kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv\") pod \"network-check-target-n62q6\" (UID: \"00006af4-11b4-46e0-afdd-71f770964dc5\") " pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:32.154268 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:32.153615 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:32.154268 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:32.153727 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 00:09:32.154268 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:32.153779 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs podName:a8e8f787-142b-4f26-9a75-9cc3b6976f35 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:40.153758406 +0000 UTC m=+18.090123763 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs") pod "network-metrics-daemon-td7ll" (UID: "a8e8f787-142b-4f26-9a75-9cc3b6976f35") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 00:09:32.154268 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:32.154151 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 00:09:32.154268 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:32.154175 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 00:09:32.154268 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:32.154188 2558 projected.go:196] Error preparing data for projected volume kube-api-access-dtzvv for pod openshift-network-diagnostics/network-check-target-n62q6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 00:09:32.154268 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:32.154235 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv podName:00006af4-11b4-46e0-afdd-71f770964dc5 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:40.154219812 +0000 UTC m=+18.090585172 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dtzvv" (UniqueName: "kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv") pod "network-check-target-n62q6" (UID: "00006af4-11b4-46e0-afdd-71f770964dc5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 00:09:32.254202 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:32.254168 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:32.254372 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:32.254316 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 00:09:32.254372 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:32.254371 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret podName:7ef46745-1efb-44fe-8511-59c406acd812 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:40.25435488 +0000 UTC m=+18.190720238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret") pod "global-pull-secret-syncer-qb4z2" (UID: "7ef46745-1efb-44fe-8511-59c406acd812") : object "kube-system"/"original-pull-secret" not registered Jul 05 00:09:32.618856 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:32.618775 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:32.619037 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:32.618875 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:32.619237 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:32.619222 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:32.619345 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:32.619325 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:32.619399 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:32.619377 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:32.619454 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:32.619440 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:34.621302 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:34.620524 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:34.621302 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:34.620655 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:34.621302 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:34.621049 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:34.621302 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:34.621136 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:34.621302 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:34.621187 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:34.621302 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:34.621254 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:36.618252 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:36.618225 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:36.618252 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:36.618253 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:36.618689 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:36.618311 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:36.618689 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:36.618422 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:36.618689 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:36.618500 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:36.618689 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:36.618658 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:38.618623 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:38.618583 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:38.618623 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:38.618617 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:38.619137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:38.618592 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:38.619137 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:38.618716 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:38.619137 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:38.618800 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:38.619137 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:38.618877 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:40.214611 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:40.214570 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtzvv\" (UniqueName: \"kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv\") pod \"network-check-target-n62q6\" (UID: \"00006af4-11b4-46e0-afdd-71f770964dc5\") " pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:40.215112 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:40.214644 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:40.215112 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:40.214751 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 00:09:40.215112 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:40.214769 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 00:09:40.215112 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:40.214790 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 00:09:40.215112 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:40.214800 2558 projected.go:196] Error preparing data for projected volume kube-api-access-dtzvv for pod openshift-network-diagnostics/network-check-target-n62q6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 00:09:40.215112 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:40.214806 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs podName:a8e8f787-142b-4f26-9a75-9cc3b6976f35 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:56.214790969 +0000 UTC m=+34.151156326 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs") pod "network-metrics-daemon-td7ll" (UID: "a8e8f787-142b-4f26-9a75-9cc3b6976f35") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 00:09:40.215112 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:40.214838 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv podName:00006af4-11b4-46e0-afdd-71f770964dc5 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:56.214827196 +0000 UTC m=+34.151192552 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dtzvv" (UniqueName: "kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv") pod "network-check-target-n62q6" (UID: "00006af4-11b4-46e0-afdd-71f770964dc5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 00:09:40.315661 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:40.315624 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:40.315825 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:40.315739 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 00:09:40.315825 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:40.315802 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret podName:7ef46745-1efb-44fe-8511-59c406acd812 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:56.315785406 +0000 UTC m=+34.252150765 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret") pod "global-pull-secret-syncer-qb4z2" (UID: "7ef46745-1efb-44fe-8511-59c406acd812") : object "kube-system"/"original-pull-secret" not registered Jul 05 00:09:40.618845 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:40.618755 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:40.618845 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:40.618781 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:40.619051 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:40.618867 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:40.619051 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:40.618887 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:40.619051 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:40.618964 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:40.619051 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:40.619035 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:42.619341 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:42.619308 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:42.619815 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:42.619425 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:42.619815 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:42.619437 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:42.619815 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:42.619547 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:42.619815 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:42.619574 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:42.619815 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:42.619617 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:44.618061 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:44.618017 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:44.618587 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:44.618024 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:44.618587 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:44.618162 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:44.618587 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:44.618024 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:44.618587 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:44.618232 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:44.618587 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:44.618308 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:44.834816 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:44.834515 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" event={"ID":"0201eff2-4a8d-4281-954f-137b7b3a8a4d","Type":"ContainerStarted","Data":"bd1676fa5a9f6976cfe8cbae29f47f6a71f51473e4956ac11844efb941d5f4d7"} Jul 05 00:09:44.836835 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:44.836660 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qlbnt" event={"ID":"fa9b29e8-905f-4113-9dd6-cde4021610d8","Type":"ContainerStarted","Data":"69fab45711f651dc0e9286ef25924d4e42d89142466d8cc06e2895462194632f"} Jul 05 00:09:44.845666 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:44.845178 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mxnwd" event={"ID":"eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1","Type":"ContainerStarted","Data":"a30eafb0363d4c48629be0afef73221fd75fdf960f2cff49814b25af757392bc"} Jul 05 00:09:44.848468 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:44.848008 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8wn9" event={"ID":"039c4612-e8d2-4bf8-994f-37600257cc9b","Type":"ContainerStarted","Data":"4fd1013b084205dbb105e6b5a51c72ef92d8f46030a14bea67ad6c1ce9c516d8"} Jul 05 00:09:44.852679 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:44.851658 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-0.ec2.internal" podStartSLOduration=20.851642081 podStartE2EDuration="20.851642081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:27.807752079 +0000 UTC m=+5.744117460" watchObservedRunningTime="2026-07-05 00:09:44.851642081 +0000 UTC m=+22.788007462" Jul 05 00:09:44.853112 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:44.853065 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2g6qj" podStartSLOduration=3.9102567969999997 podStartE2EDuration="22.853050944s" totalImagesPullingTime="18.942794147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:44.851879549 +0000 UTC m=+22.788244929" watchObservedRunningTime="2026-07-05 00:09:44.853050944 +0000 UTC m=+22.789416324" Jul 05 00:09:44.854182 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:44.854121 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" event={"ID":"a828f18f-4034-4e6f-963f-04e391abbe14","Type":"ContainerStarted","Data":"dc1b9b07d722179910ab5b14199031435368fbcf884aee2dfc85562f0219ef2f"} Jul 05 00:09:44.889194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:44.889150 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mxnwd" podStartSLOduration=2.9026348840000002 podStartE2EDuration="21.889135752s" totalImagesPullingTime="18.986500868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:44.88873662 +0000 UTC m=+22.825101999" watchObservedRunningTime="2026-07-05 00:09:44.889135752 +0000 UTC m=+22.825501131" Jul 05 00:09:44.903794 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:44.903756 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qlbnt" podStartSLOduration=11.034117543 podStartE2EDuration="21.903741856s" totalImagesPullingTime="10.869624313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:44.903722288 +0000 UTC m=+22.840087667" watchObservedRunningTime="2026-07-05 00:09:44.903741856 +0000 UTC m=+22.840107236" Jul 05 00:09:44.918794 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:44.918741 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9595p" podStartSLOduration=2.9867803200000003 podStartE2EDuration="21.918721056s" totalImagesPullingTime="18.931940736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:44.91783842 +0000 UTC m=+22.854203800" watchObservedRunningTime="2026-07-05 00:09:44.918721056 +0000 UTC m=+22.855086436" Jul 05 00:09:45.020125 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.019970 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:45.078450 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.078417 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:45.855015 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.854971 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qlbnt" Jul 05 00:09:45.858190 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.858164 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" event={"ID":"e97fe81d-cff9-4c26-944b-030bd1d1a1c9","Type":"ContainerStarted","Data":"8d462764b98a22b11b0aed5872c8b8e76c35ebc3ddef3e4261b69a95ebd31b9e"} Jul 05 00:09:45.859667 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.859637 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8s5zw" event={"ID":"f2fde0b3-a812-4baf-9da4-719e7b227fd9","Type":"ContainerStarted","Data":"d834329640db6595afc0927ad43799346aeaf55388e0cf8c2f11c33c5cd01058"} Jul 05 00:09:45.861173 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.861152 2558 generic.go:356] "Generic (PLEG): container finished" podID="039c4612-e8d2-4bf8-994f-37600257cc9b" containerID="4fd1013b084205dbb105e6b5a51c72ef92d8f46030a14bea67ad6c1ce9c516d8" exitCode=0 Jul 05 00:09:45.861292 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.861213 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8wn9" event={"ID":"039c4612-e8d2-4bf8-994f-37600257cc9b","Type":"ContainerDied","Data":"4fd1013b084205dbb105e6b5a51c72ef92d8f46030a14bea67ad6c1ce9c516d8"} Jul 05 00:09:45.862786 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.862770 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9595p" event={"ID":"215cde65-c249-45c8-b941-488c4ab6b568","Type":"ContainerStarted","Data":"0d652b665539a869dcd9552f5366f25b289a39ddd4a5f18a52512af2d3d511ee"} Jul 05 00:09:45.866724 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.866591 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" event={"ID":"a828f18f-4034-4e6f-963f-04e391abbe14","Type":"ContainerStarted","Data":"d7407795a77a1d0dcfed51de0fc20bd50ac440b3e6e4bed3bbf36ac8a4e9712a"} Jul 05 00:09:45.867347 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.867322 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" event={"ID":"a828f18f-4034-4e6f-963f-04e391abbe14","Type":"ContainerStarted","Data":"216d24781ebc377220b8a10b96b43e12bfca6eddd8d287cf437079231d0328be"} Jul 05 00:09:45.867752 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.867728 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" event={"ID":"a828f18f-4034-4e6f-963f-04e391abbe14","Type":"ContainerStarted","Data":"5624b6eb24924c49866b613f011155c6c3958771d4e51d6d5ba2b2e2b0c0d235"} Jul 05 00:09:45.867976 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.867957 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" event={"ID":"a828f18f-4034-4e6f-963f-04e391abbe14","Type":"ContainerStarted","Data":"639f272e5197a943923ba4114ce5db6f64b2b569afaa9a29e42965732f34da7f"} Jul 05 00:09:45.868261 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.868241 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" event={"ID":"a828f18f-4034-4e6f-963f-04e391abbe14","Type":"ContainerStarted","Data":"581c97747770f674650d40254164e8eed5f626c50c0f9f84e90fa24a5dfda65e"} Jul 05 00:09:45.892826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.892750 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8s5zw" podStartSLOduration=3.957857916 podStartE2EDuration="22.892736627s" totalImagesPullingTime="18.934878711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:45.873556171 +0000 UTC m=+23.809921552" watchObservedRunningTime="2026-07-05 00:09:45.892736627 +0000 UTC m=+23.829102004" Jul 05 00:09:45.928303 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:45.928234 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:46.031436 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:46.031401 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:46.346809 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:46.346784 2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 05 00:09:46.545741 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:46.545590 2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-05T00:09:46.346802002Z","UUID":"7beb3c54-c037-4ac4-b2fa-1fe6dccc0271","Handler":null,"Name":"","Endpoint":""} Jul 05 00:09:46.547250 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:46.547227 2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 05 00:09:46.547369 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:46.547273 2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 05 00:09:46.618683 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:46.618655 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:46.618683 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:46.618673 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:46.618954 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:46.618659 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:46.618954 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:46.618781 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:46.618954 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:46.618876 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:46.619081 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:46.618967 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:46.871552 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:46.871476 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" event={"ID":"e97fe81d-cff9-4c26-944b-030bd1d1a1c9","Type":"ContainerStarted","Data":"03d5a1ed2dd7f667146600dcc4b5b36a9b7aae7e207b052291896852d7f233b0"} Jul 05 00:09:46.873399 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:46.873289 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qd79j" event={"ID":"a45b95a1-e42e-406b-9df7-096497d8369e","Type":"ContainerStarted","Data":"84bcf5246125f31b737171495fef311eaa5af0c51b2fcf723510ea37fc5bb94e"} Jul 05 00:09:46.888391 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:46.888331 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qd79j" podStartSLOduration=4.9618810920000005 podStartE2EDuration="23.88831229s" totalImagesPullingTime="18.926431198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:46.887933133 +0000 UTC m=+24.824298511" watchObservedRunningTime="2026-07-05 00:09:46.88831229 +0000 UTC m=+24.824677673" Jul 05 00:09:46.992234 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:46.992195 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:47.063232 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:47.063202 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:47.165516 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:47.165478 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:47.553363 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:47.553337 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qlbnt" Jul 05 00:09:47.553828 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:47.553814 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qlbnt" Jul 05 00:09:47.879034 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:47.878742 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" event={"ID":"e97fe81d-cff9-4c26-944b-030bd1d1a1c9","Type":"ContainerStarted","Data":"dd849a4eee57c66c8d4cb5db49a7fa3746c0988aad6806bae22ca6a9a625fa04"} Jul 05 00:09:47.882606 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:47.882571 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" event={"ID":"a828f18f-4034-4e6f-963f-04e391abbe14","Type":"ContainerStarted","Data":"79b8f0c045e57cb45961450157043649f9eff7ba0d43369806f190a2a1dd21d1"} Jul 05 00:09:47.883478 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:47.883455 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qlbnt" Jul 05 00:09:47.907846 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:47.907797 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gbbq" podStartSLOduration=3.292063171 podStartE2EDuration="24.907784722s" totalImagesPullingTime="21.615721551s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:47.894606198 +0000 UTC m=+25.830971575" watchObservedRunningTime="2026-07-05 00:09:47.907784722 +0000 UTC m=+25.844150100" Jul 05 00:09:47.945489 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:47.945418 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:48.618156 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:48.618125 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:48.618382 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:48.618125 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:48.618382 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:48.618223 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:48.618382 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:48.618127 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:48.618382 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:48.618323 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:48.618382 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:48.618365 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:50.618552 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:50.618510 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:50.619245 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:50.618658 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:50.619245 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:50.619086 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:50.619245 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:50.619195 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:50.619391 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:50.619314 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:50.619441 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:50.619394 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:51.058988 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:51.058948 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:51.110944 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:51.110774 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:51.402360 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:51.402328 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:51.453975 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:51.453894 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:51.893153 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:51.893125 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" event={"ID":"a828f18f-4034-4e6f-963f-04e391abbe14","Type":"ContainerStarted","Data":"1290a19780e4cfb097882b15dd9f51fb68549db760cd84f33a4bc19345469860"} Jul 05 00:09:51.893739 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:51.893372 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:51.893739 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:51.893436 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:51.893739 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:51.893478 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:51.895191 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:51.895159 2558 generic.go:356] "Generic (PLEG): container finished" podID="039c4612-e8d2-4bf8-994f-37600257cc9b" containerID="784fa67e8679d03f6f272505e09dccabe97fef21c06e7baa4045238ec590be76" exitCode=0 Jul 05 00:09:51.895191 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:51.895181 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8wn9" event={"ID":"039c4612-e8d2-4bf8-994f-37600257cc9b","Type":"ContainerDied","Data":"784fa67e8679d03f6f272505e09dccabe97fef21c06e7baa4045238ec590be76"} Jul 05 00:09:51.909637 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:51.909617 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:51.909766 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:51.909744 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:09:51.917807 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:51.917768 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" podStartSLOduration=9.903110562 podStartE2EDuration="28.917756294s" totalImagesPullingTime="19.014645732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:09:51.916095013 +0000 UTC m=+29.852460391" watchObservedRunningTime="2026-07-05 00:09:51.917756294 +0000 UTC m=+29.854121674" Jul 05 00:09:52.623428 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:52.622518 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:52.623428 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:52.622715 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:52.623428 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:52.622875 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:52.623428 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:52.623027 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:52.623428 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:52.623125 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:52.623856 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:52.623414 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:52.931152 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:52.930993 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qb4z2"] Jul 05 00:09:52.931464 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:52.931211 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:52.931464 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:52.931285 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:52.934292 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:52.934265 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-td7ll"] Jul 05 00:09:52.934401 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:52.934369 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:52.934505 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:52.934485 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:52.935052 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:52.935032 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n62q6"] Jul 05 00:09:52.935146 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:52.935108 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:52.935212 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:52.935194 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:53.608317 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:53.608285 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:53.656166 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:53.656140 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:53.900809 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:53.900780 2558 generic.go:356] "Generic (PLEG): container finished" podID="039c4612-e8d2-4bf8-994f-37600257cc9b" containerID="8886602b77ed21509244bbbc82aa1b3ee699d218d23e0690af9444e3534f2034" exitCode=0 Jul 05 00:09:53.900997 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:53.900872 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8wn9" event={"ID":"039c4612-e8d2-4bf8-994f-37600257cc9b","Type":"ContainerDied","Data":"8886602b77ed21509244bbbc82aa1b3ee699d218d23e0690af9444e3534f2034"} Jul 05 00:09:54.618804 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:54.618773 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:54.619347 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:54.618869 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:54.619347 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:54.618773 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:54.619347 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:54.618925 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:54.619347 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:54.618965 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:54.619347 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:54.619007 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:55.906958 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:55.906927 2558 generic.go:356] "Generic (PLEG): container finished" podID="039c4612-e8d2-4bf8-994f-37600257cc9b" containerID="cd860d0d4d94890db9364a1df8e60ab059543bd649682976dfd521fdbff970d4" exitCode=0 Jul 05 00:09:55.907476 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:55.906972 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8wn9" event={"ID":"039c4612-e8d2-4bf8-994f-37600257cc9b","Type":"ContainerDied","Data":"cd860d0d4d94890db9364a1df8e60ab059543bd649682976dfd521fdbff970d4"} Jul 05 00:09:56.231790 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:56.231701 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtzvv\" (UniqueName: \"kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv\") pod \"network-check-target-n62q6\" (UID: \"00006af4-11b4-46e0-afdd-71f770964dc5\") " pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:56.231790 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:56.231756 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:56.232014 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:56.231850 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 00:09:56.232014 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:56.231857 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 00:09:56.232014 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:56.231877 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 00:09:56.232014 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:56.231887 2558 projected.go:196] Error preparing data for projected volume kube-api-access-dtzvv for pod openshift-network-diagnostics/network-check-target-n62q6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 00:09:56.232014 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:56.231946 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs podName:a8e8f787-142b-4f26-9a75-9cc3b6976f35 nodeName:}" failed. No retries permitted until 2026-07-05 00:10:28.231891158 +0000 UTC m=+66.168256517 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs") pod "network-metrics-daemon-td7ll" (UID: "a8e8f787-142b-4f26-9a75-9cc3b6976f35") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 00:09:56.232201 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:56.232029 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv podName:00006af4-11b4-46e0-afdd-71f770964dc5 nodeName:}" failed. No retries permitted until 2026-07-05 00:10:28.232016054 +0000 UTC m=+66.168381410 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dtzvv" (UniqueName: "kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv") pod "network-check-target-n62q6" (UID: "00006af4-11b4-46e0-afdd-71f770964dc5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 00:09:56.332253 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:56.332219 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:56.332410 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:56.332376 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 00:09:56.332466 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:56.332438 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret podName:7ef46745-1efb-44fe-8511-59c406acd812 nodeName:}" failed. No retries permitted until 2026-07-05 00:10:28.332424311 +0000 UTC m=+66.268789672 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret") pod "global-pull-secret-syncer-qb4z2" (UID: "7ef46745-1efb-44fe-8511-59c406acd812") : object "kube-system"/"original-pull-secret" not registered Jul 05 00:09:56.618012 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:56.617926 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:56.618012 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:56.617967 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:56.618200 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:56.617936 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:56.618200 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:56.618055 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qb4z2" podUID="7ef46745-1efb-44fe-8511-59c406acd812" Jul 05 00:09:56.618200 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:56.618145 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n62q6" podUID="00006af4-11b4-46e0-afdd-71f770964dc5" Jul 05 00:09:56.618343 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:56.618242 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:09:57.347077 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.347045 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-0.ec2.internal" event="NodeReady" Jul 05 00:09:57.347509 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.347174 2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 05 00:09:57.391052 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.391024 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-s764k"] Jul 05 00:09:57.394539 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.394518 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-c9fwj"] Jul 05 00:09:57.394704 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.394643 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s764k" Jul 05 00:09:57.397059 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.397034 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 05 00:09:57.397227 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.397205 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-z9vd9\"" Jul 05 00:09:57.397334 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.397227 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 05 00:09:57.398778 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.398763 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c9fwj" Jul 05 00:09:57.401714 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.401227 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 05 00:09:57.401714 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.401250 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ch6vj\"" Jul 05 00:09:57.401714 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.401258 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 05 00:09:57.401714 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.401552 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 05 00:09:57.405951 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.405929 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-s764k"] Jul 05 00:09:57.406352 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.406332 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c9fwj"] Jul 05 00:09:57.498798 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.498750 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:57.540518 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.540489 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4hcp\" (UniqueName: \"kubernetes.io/projected/a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4-kube-api-access-s4hcp\") pod \"ingress-canary-c9fwj\" (UID: \"a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4\") " pod="openshift-ingress-canary/ingress-canary-c9fwj" Jul 05 00:09:57.540518 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.540523 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-tmp-dir\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:09:57.540743 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.540567 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:09:57.540743 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.540628 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp7qt\" (UniqueName: \"kubernetes.io/projected/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-kube-api-access-gp7qt\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:09:57.540743 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.540717 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-config-volume\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:09:57.540894 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.540754 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4-cert\") pod \"ingress-canary-c9fwj\" (UID: \"a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4\") " pod="openshift-ingress-canary/ingress-canary-c9fwj" Jul 05 00:09:57.549947 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.549917 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:09:57.641390 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.641355 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp7qt\" (UniqueName: \"kubernetes.io/projected/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-kube-api-access-gp7qt\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:09:57.641568 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.641405 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-config-volume\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:09:57.641568 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.641429 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4-cert\") pod \"ingress-canary-c9fwj\" (UID: \"a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4\") " pod="openshift-ingress-canary/ingress-canary-c9fwj" Jul 05 00:09:57.641568 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.641457 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4hcp\" (UniqueName: \"kubernetes.io/projected/a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4-kube-api-access-s4hcp\") pod \"ingress-canary-c9fwj\" (UID: \"a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4\") " pod="openshift-ingress-canary/ingress-canary-c9fwj" Jul 05 00:09:57.641568 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.641486 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-tmp-dir\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:09:57.641568 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.641532 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:09:57.641857 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:57.641623 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 00:09:57.641857 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:57.641681 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls podName:dd7a485d-b0d9-4e77-bf59-bdf7ed11f365 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:58.141663136 +0000 UTC m=+36.078028497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls") pod "dns-default-s764k" (UID: "dd7a485d-b0d9-4e77-bf59-bdf7ed11f365") : secret "dns-default-metrics-tls" not found Jul 05 00:09:57.641857 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.641830 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-tmp-dir\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:09:57.642043 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.642024 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-config-volume\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:09:57.645761 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.645743 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4-cert\") pod \"ingress-canary-c9fwj\" (UID: \"a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4\") " pod="openshift-ingress-canary/ingress-canary-c9fwj" Jul 05 00:09:57.650057 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.650035 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp7qt\" (UniqueName: \"kubernetes.io/projected/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-kube-api-access-gp7qt\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:09:57.651211 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.651193 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4hcp\" (UniqueName: \"kubernetes.io/projected/a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4-kube-api-access-s4hcp\") pod \"ingress-canary-c9fwj\" (UID: \"a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4\") " pod="openshift-ingress-canary/ingress-canary-c9fwj" Jul 05 00:09:57.710374 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.710309 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c9fwj" Jul 05 00:09:57.864753 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.864561 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c9fwj"] Jul 05 00:09:57.869304 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:09:57.869278 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9df4b2b_5e36_47e0_a1eb_fe2bd1aa44a4.slice/crio-86fffd285a54e2e719618334228cdab8dab2816676eb7d1a8be4e6c70b6043ec WatchSource:0}: Error finding container 86fffd285a54e2e719618334228cdab8dab2816676eb7d1a8be4e6c70b6043ec: Status 404 returned error can't find the container with id 86fffd285a54e2e719618334228cdab8dab2816676eb7d1a8be4e6c70b6043ec Jul 05 00:09:57.913380 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:57.913341 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c9fwj" event={"ID":"a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4","Type":"ContainerStarted","Data":"86fffd285a54e2e719618334228cdab8dab2816676eb7d1a8be4e6c70b6043ec"} Jul 05 00:09:58.146783 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:58.146745 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:09:58.146960 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:58.146922 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 00:09:58.147027 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:58.147001 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls podName:dd7a485d-b0d9-4e77-bf59-bdf7ed11f365 nodeName:}" failed. No retries permitted until 2026-07-05 00:09:59.146979237 +0000 UTC m=+37.083344597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls") pod "dns-default-s764k" (UID: "dd7a485d-b0d9-4e77-bf59-bdf7ed11f365") : secret "dns-default-metrics-tls" not found Jul 05 00:09:58.618033 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:58.617954 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:09:58.618033 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:58.617993 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:09:58.618550 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:58.617957 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:09:58.621050 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:58.621027 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 00:09:58.621179 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:58.621027 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fmjx4\"" Jul 05 00:09:58.621179 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:58.621026 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 05 00:09:58.622423 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:58.622155 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 00:09:58.622423 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:58.622251 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m9r6w\"" Jul 05 00:09:58.622423 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:58.622362 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 05 00:09:59.155417 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:09:59.155386 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:09:59.155589 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:59.155552 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 00:09:59.155665 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:09:59.155623 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls podName:dd7a485d-b0d9-4e77-bf59-bdf7ed11f365 nodeName:}" failed. No retries permitted until 2026-07-05 00:10:01.155604617 +0000 UTC m=+39.091969988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls") pod "dns-default-s764k" (UID: "dd7a485d-b0d9-4e77-bf59-bdf7ed11f365") : secret "dns-default-metrics-tls" not found Jul 05 00:10:01.172574 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:01.172542 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:10:01.173009 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:10:01.172669 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 00:10:01.173009 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:10:01.172749 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls podName:dd7a485d-b0d9-4e77-bf59-bdf7ed11f365 nodeName:}" failed. No retries permitted until 2026-07-05 00:10:05.172732163 +0000 UTC m=+43.109097522 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls") pod "dns-default-s764k" (UID: "dd7a485d-b0d9-4e77-bf59-bdf7ed11f365") : secret "dns-default-metrics-tls" not found Jul 05 00:10:01.923850 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:01.923646 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c9fwj" event={"ID":"a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4","Type":"ContainerStarted","Data":"ff5c4088c77543c93966416d847352729c0fb61c29c3588ad608c99fb29b6da7"} Jul 05 00:10:01.926237 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:01.926214 2558 generic.go:356] "Generic (PLEG): container finished" podID="039c4612-e8d2-4bf8-994f-37600257cc9b" containerID="ed5a05fbe5aeb6afd1854ae399cf803d8cda37dbc062e08f3ada8ab902cadfb5" exitCode=0 Jul 05 00:10:01.926345 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:01.926243 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8wn9" event={"ID":"039c4612-e8d2-4bf8-994f-37600257cc9b","Type":"ContainerDied","Data":"ed5a05fbe5aeb6afd1854ae399cf803d8cda37dbc062e08f3ada8ab902cadfb5"} Jul 05 00:10:01.938944 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:01.938886 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-c9fwj" podStartSLOduration=1.290905559 podStartE2EDuration="4.938873205s" totalImagesPullingTime="3.647967646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:10:01.937846079 +0000 UTC m=+39.874211458" watchObservedRunningTime="2026-07-05 00:10:01.938873205 +0000 UTC m=+39.875238583" Jul 05 00:10:02.406863 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:02.406831 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:10:02.712207 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:02.712133 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:10:02.931224 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:02.931191 2558 generic.go:356] "Generic (PLEG): container finished" podID="039c4612-e8d2-4bf8-994f-37600257cc9b" containerID="782714ae12f953ac06c7bdcb79ef409e854dc7fff6c70d6fe9aa7aadce2c5ec7" exitCode=0 Jul 05 00:10:02.931393 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:02.931279 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8wn9" event={"ID":"039c4612-e8d2-4bf8-994f-37600257cc9b","Type":"ContainerDied","Data":"782714ae12f953ac06c7bdcb79ef409e854dc7fff6c70d6fe9aa7aadce2c5ec7"} Jul 05 00:10:03.939039 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:03.938999 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8wn9" event={"ID":"039c4612-e8d2-4bf8-994f-37600257cc9b","Type":"ContainerStarted","Data":"6013b82f76dc31c56fc4210f7ddc32e3b25f7a4990eb89eee8eff84ab1558f50"} Jul 05 00:10:03.960578 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:03.960535 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-k8wn9" podStartSLOduration=9.120191216 podStartE2EDuration="40.96052193s" totalImagesPullingTime="31.840330714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:10:03.959080034 +0000 UTC m=+41.895445414" watchObservedRunningTime="2026-07-05 00:10:03.96052193 +0000 UTC m=+41.896887309" Jul 05 00:10:04.083074 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:04.083041 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:10:04.133987 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:04.133958 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:10:05.202754 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:05.202722 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:10:05.203132 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:10:05.202813 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 00:10:05.203132 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:10:05.202863 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls podName:dd7a485d-b0d9-4e77-bf59-bdf7ed11f365 nodeName:}" failed. No retries permitted until 2026-07-05 00:10:13.20285049 +0000 UTC m=+51.139215848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls") pod "dns-default-s764k" (UID: "dd7a485d-b0d9-4e77-bf59-bdf7ed11f365") : secret "dns-default-metrics-tls" not found Jul 05 00:10:13.256005 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:13.255968 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:10:13.256479 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:10:13.256102 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 00:10:13.256479 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:10:13.256156 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls podName:dd7a485d-b0d9-4e77-bf59-bdf7ed11f365 nodeName:}" failed. No retries permitted until 2026-07-05 00:10:29.256142947 +0000 UTC m=+67.192508303 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls") pod "dns-default-s764k" (UID: "dd7a485d-b0d9-4e77-bf59-bdf7ed11f365") : secret "dns-default-metrics-tls" not found Jul 05 00:10:13.591498 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:13.591418 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:10:13.691799 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:13.691773 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:10:23.915074 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:23.915048 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-56fqd" Jul 05 00:10:24.015680 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.015648 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:10:24.117798 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.117769 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:10:24.169843 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.169781 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:10:24.455264 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.455184 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj"] Jul 05 00:10:24.457722 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.457701 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.460696 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.460601 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 05 00:10:24.460696 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.460628 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 05 00:10:24.460696 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.460646 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 05 00:10:24.460696 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.460657 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 05 00:10:24.460929 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.460707 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 05 00:10:24.460929 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.460728 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 05 00:10:24.460996 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.460974 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 05 00:10:24.472345 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.469263 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj"] Jul 05 00:10:24.635257 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.635222 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/528a0a1e-b618-4606-a7ae-c78c01c8232b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.635257 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.635254 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d9dw\" (UniqueName: \"kubernetes.io/projected/528a0a1e-b618-4606-a7ae-c78c01c8232b-kube-api-access-8d9dw\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.635491 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.635281 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/528a0a1e-b618-4606-a7ae-c78c01c8232b-hub\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.635491 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.635360 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/528a0a1e-b618-4606-a7ae-c78c01c8232b-ca\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.635491 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.635416 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/528a0a1e-b618-4606-a7ae-c78c01c8232b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.635491 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.635446 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/528a0a1e-b618-4606-a7ae-c78c01c8232b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.736795 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.736733 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/528a0a1e-b618-4606-a7ae-c78c01c8232b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.736795 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.736763 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d9dw\" (UniqueName: \"kubernetes.io/projected/528a0a1e-b618-4606-a7ae-c78c01c8232b-kube-api-access-8d9dw\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.736795 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.736782 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/528a0a1e-b618-4606-a7ae-c78c01c8232b-hub\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.737041 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.736799 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/528a0a1e-b618-4606-a7ae-c78c01c8232b-ca\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.737041 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.736842 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/528a0a1e-b618-4606-a7ae-c78c01c8232b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.737041 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.736867 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/528a0a1e-b618-4606-a7ae-c78c01c8232b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.737597 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.737569 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/528a0a1e-b618-4606-a7ae-c78c01c8232b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.740890 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.740863 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/528a0a1e-b618-4606-a7ae-c78c01c8232b-ca\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.740999 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.740988 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/528a0a1e-b618-4606-a7ae-c78c01c8232b-hub\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.741036 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.740998 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/528a0a1e-b618-4606-a7ae-c78c01c8232b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.741070 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.741045 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/528a0a1e-b618-4606-a7ae-c78c01c8232b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.744869 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.744841 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d9dw\" (UniqueName: \"kubernetes.io/projected/528a0a1e-b618-4606-a7ae-c78c01c8232b-kube-api-access-8d9dw\") pod \"cluster-proxy-proxy-agent-5d8457b789-zx6wj\" (UID: \"528a0a1e-b618-4606-a7ae-c78c01c8232b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.774692 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.774670 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:10:24.897242 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.897213 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj"] Jul 05 00:10:24.900494 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:10:24.900461 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod528a0a1e_b618_4606_a7ae_c78c01c8232b.slice/crio-e290b9444794f0a27a4e808734991ea73291191373851775d7ae5d1a968199e2 WatchSource:0}: Error finding container e290b9444794f0a27a4e808734991ea73291191373851775d7ae5d1a968199e2: Status 404 returned error can't find the container with id e290b9444794f0a27a4e808734991ea73291191373851775d7ae5d1a968199e2 Jul 05 00:10:24.986266 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:24.986237 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" event={"ID":"528a0a1e-b618-4606-a7ae-c78c01c8232b","Type":"ContainerStarted","Data":"e290b9444794f0a27a4e808734991ea73291191373851775d7ae5d1a968199e2"} Jul 05 00:10:28.264852 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.264812 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:10:28.265291 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.264897 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtzvv\" (UniqueName: \"kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv\") pod \"network-check-target-n62q6\" (UID: \"00006af4-11b4-46e0-afdd-71f770964dc5\") " pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:10:28.267550 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.267520 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 00:10:28.267658 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.267564 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 05 00:10:28.275738 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:10:28.275719 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 05 00:10:28.275847 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:10:28.275778 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs podName:a8e8f787-142b-4f26-9a75-9cc3b6976f35 nodeName:}" failed. No retries permitted until 2026-07-05 00:11:32.275759113 +0000 UTC m=+130.212124484 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs") pod "network-metrics-daemon-td7ll" (UID: "a8e8f787-142b-4f26-9a75-9cc3b6976f35") : secret "metrics-daemon-secret" not found Jul 05 00:10:28.278034 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.278007 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 05 00:10:28.289106 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.289084 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtzvv\" (UniqueName: \"kubernetes.io/projected/00006af4-11b4-46e0-afdd-71f770964dc5-kube-api-access-dtzvv\") pod \"network-check-target-n62q6\" (UID: \"00006af4-11b4-46e0-afdd-71f770964dc5\") " pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:10:28.357843 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.357810 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fmjx4\"" Jul 05 00:10:28.366017 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.365987 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:10:28.366214 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.366198 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:10:28.368726 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.368711 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 00:10:28.378583 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.378555 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ef46745-1efb-44fe-8511-59c406acd812-original-pull-secret\") pod \"global-pull-secret-syncer-qb4z2\" (UID: \"7ef46745-1efb-44fe-8511-59c406acd812\") " pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:10:28.497624 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.497596 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n62q6"] Jul 05 00:10:28.501464 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:10:28.501437 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00006af4_11b4_46e0_afdd_71f770964dc5.slice/crio-84abac931ba27caca876e417559eb8d59d380a2af063cf3b3e0cd9834b8a4cc5 WatchSource:0}: Error finding container 84abac931ba27caca876e417559eb8d59d380a2af063cf3b3e0cd9834b8a4cc5: Status 404 returned error can't find the container with id 84abac931ba27caca876e417559eb8d59d380a2af063cf3b3e0cd9834b8a4cc5 Jul 05 00:10:28.641568 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.641541 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qb4z2" Jul 05 00:10:28.776291 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.776198 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qb4z2"] Jul 05 00:10:28.779787 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:10:28.779764 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef46745_1efb_44fe_8511_59c406acd812.slice/crio-ba61621cd2935f1f02a6d878df38662e482d3ae7a4405e1e569b8459a9424126 WatchSource:0}: Error finding container ba61621cd2935f1f02a6d878df38662e482d3ae7a4405e1e569b8459a9424126: Status 404 returned error can't find the container with id ba61621cd2935f1f02a6d878df38662e482d3ae7a4405e1e569b8459a9424126 Jul 05 00:10:28.998861 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:28.998819 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qb4z2" event={"ID":"7ef46745-1efb-44fe-8511-59c406acd812","Type":"ContainerStarted","Data":"ba61621cd2935f1f02a6d878df38662e482d3ae7a4405e1e569b8459a9424126"} Jul 05 00:10:29.001425 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:29.001397 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" event={"ID":"528a0a1e-b618-4606-a7ae-c78c01c8232b","Type":"ContainerStarted","Data":"aeb68deee4f3ad7f2643f1321e0fb920e39c1c2417439a3e1da0d55ecf21a45d"} Jul 05 00:10:29.001552 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:29.001433 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" event={"ID":"528a0a1e-b618-4606-a7ae-c78c01c8232b","Type":"ContainerStarted","Data":"60308e8aa45e2c69eecce4894aa6716118dd85438e8df6602f930fe13fde5015"} Jul 05 00:10:29.001552 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:29.001455 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" event={"ID":"528a0a1e-b618-4606-a7ae-c78c01c8232b","Type":"ContainerStarted","Data":"852c7d26f1dc6e31467efa4ff3118bd26b34346796adab0c5d031e02a2dc08e2"} Jul 05 00:10:29.002662 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:29.002636 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n62q6" event={"ID":"00006af4-11b4-46e0-afdd-71f770964dc5","Type":"ContainerStarted","Data":"84abac931ba27caca876e417559eb8d59d380a2af063cf3b3e0cd9834b8a4cc5"} Jul 05 00:10:29.025836 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:29.025778 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" podStartSLOduration=1.555715033 podStartE2EDuration="5.025759308s" totalImagesPullingTime="3.470044275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:10:29.025224622 +0000 UTC m=+66.961590002" watchObservedRunningTime="2026-07-05 00:10:29.025759308 +0000 UTC m=+66.962124686" Jul 05 00:10:29.273504 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:29.273472 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:10:29.274013 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:10:29.273656 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 00:10:29.274013 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:10:29.273736 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls podName:dd7a485d-b0d9-4e77-bf59-bdf7ed11f365 nodeName:}" failed. No retries permitted until 2026-07-05 00:11:01.273716191 +0000 UTC m=+99.210081548 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls") pod "dns-default-s764k" (UID: "dd7a485d-b0d9-4e77-bf59-bdf7ed11f365") : secret "dns-default-metrics-tls" not found Jul 05 00:10:34.020711 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:34.020671 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qb4z2" event={"ID":"7ef46745-1efb-44fe-8511-59c406acd812","Type":"ContainerStarted","Data":"355fb467ebe9234780203432c98db592a2cf887381330c0e04ed96d890bb7c58"} Jul 05 00:10:34.022167 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:34.022142 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n62q6" event={"ID":"00006af4-11b4-46e0-afdd-71f770964dc5","Type":"ContainerStarted","Data":"beb8f8f93cf1d774a6a0aa5a5a8ed05246d75c92e15d2e43fcea88fb0388f77f"} Jul 05 00:10:34.022274 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:34.022260 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:10:34.036882 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:34.035318 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qb4z2" podStartSLOduration=66.586847895 podStartE2EDuration="1m11.035299808s" totalImagesPullingTime="4.448451913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:10:34.034030312 +0000 UTC m=+71.970395692" watchObservedRunningTime="2026-07-05 00:10:34.035299808 +0000 UTC m=+71.971665189" Jul 05 00:10:52.264035 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:10:52.263886 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:11:01.285620 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:11:01.285586 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:11:01.286007 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:11:01.285706 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 00:11:01.286007 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:11:01.285771 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls podName:dd7a485d-b0d9-4e77-bf59-bdf7ed11f365 nodeName:}" failed. No retries permitted until 2026-07-05 00:12:05.285754024 +0000 UTC m=+163.222119388 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls") pod "dns-default-s764k" (UID: "dd7a485d-b0d9-4e77-bf59-bdf7ed11f365") : secret "dns-default-metrics-tls" not found Jul 05 00:11:05.027416 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:11:05.027388 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n62q6" Jul 05 00:11:05.042228 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:11:05.042181 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n62q6" podStartSLOduration=97.320136393 podStartE2EDuration="1m42.042168989s" totalImagesPullingTime="4.722032596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:10:34.047436791 +0000 UTC m=+71.983802183" watchObservedRunningTime="2026-07-05 00:11:05.042168989 +0000 UTC m=+102.978534367" Jul 05 00:11:32.286715 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:11:32.286668 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:11:32.287134 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:11:32.286791 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 05 00:11:32.287134 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:11:32.286854 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs podName:a8e8f787-142b-4f26-9a75-9cc3b6976f35 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:34.28683908 +0000 UTC m=+252.223204438 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs") pod "network-metrics-daemon-td7ll" (UID: "a8e8f787-142b-4f26-9a75-9cc3b6976f35") : secret "metrics-daemon-secret" not found Jul 05 00:12:00.404283 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:12:00.404242 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-s764k" podUID="dd7a485d-b0d9-4e77-bf59-bdf7ed11f365" Jul 05 00:12:01.259307 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:12:01.259278 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s764k" Jul 05 00:12:01.631812 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:12:01.631730 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-td7ll" podUID="a8e8f787-142b-4f26-9a75-9cc3b6976f35" Jul 05 00:12:05.300839 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:12:05.300809 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:12:05.301215 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:12:05.300955 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 05 00:12:05.301215 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:12:05.301016 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls podName:dd7a485d-b0d9-4e77-bf59-bdf7ed11f365 nodeName:}" failed. No retries permitted until 2026-07-05 00:14:07.300996211 +0000 UTC m=+285.237361568 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls") pod "dns-default-s764k" (UID: "dd7a485d-b0d9-4e77-bf59-bdf7ed11f365") : secret "dns-default-metrics-tls" not found Jul 05 00:12:12.619190 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:12:12.619110 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:12:48.351570 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:12:48.351538 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:16.306399 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.306364 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg"] Jul 05 00:13:16.308187 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.308172 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:16.310797 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.310779 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 05 00:13:16.310935 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.310893 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mzjx4\"" Jul 05 00:13:16.313647 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.313629 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 05 00:13:16.313739 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.313657 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 05 00:13:16.313739 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.313690 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 05 00:13:16.320226 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.320205 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg"] Jul 05 00:13:16.366012 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.365978 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/10b6eca4-7fae-43e3-aeec-52934b2be942-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:16.366175 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.366044 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnk96\" (UniqueName: \"kubernetes.io/projected/10b6eca4-7fae-43e3-aeec-52934b2be942-kube-api-access-rnk96\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:16.366175 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.366108 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:16.427972 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.427943 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:16.466952 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.466923 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnk96\" (UniqueName: \"kubernetes.io/projected/10b6eca4-7fae-43e3-aeec-52934b2be942-kube-api-access-rnk96\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:16.467093 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.466970 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:16.467093 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.466996 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/10b6eca4-7fae-43e3-aeec-52934b2be942-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:16.467093 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:16.467084 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 00:13:16.467196 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:16.467159 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls podName:10b6eca4-7fae-43e3-aeec-52934b2be942 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:16.967141262 +0000 UTC m=+234.903506618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tbvtg" (UID: "10b6eca4-7fae-43e3-aeec-52934b2be942") : secret "cluster-monitoring-operator-tls" not found Jul 05 00:13:16.467683 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.467664 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/10b6eca4-7fae-43e3-aeec-52934b2be942-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:16.474932 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.474894 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnk96\" (UniqueName: \"kubernetes.io/projected/10b6eca4-7fae-43e3-aeec-52934b2be942-kube-api-access-rnk96\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:16.525601 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.525578 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:16.970475 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:16.970430 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:16.970654 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:16.970583 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 00:13:16.970703 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:16.970662 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls podName:10b6eca4-7fae-43e3-aeec-52934b2be942 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:17.97064037 +0000 UTC m=+235.907005727 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tbvtg" (UID: "10b6eca4-7fae-43e3-aeec-52934b2be942") : secret "cluster-monitoring-operator-tls" not found Jul 05 00:13:17.977600 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:17.977558 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:17.978107 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:17.977725 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 00:13:17.978107 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:17.977806 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls podName:10b6eca4-7fae-43e3-aeec-52934b2be942 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:19.977786935 +0000 UTC m=+237.914152292 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tbvtg" (UID: "10b6eca4-7fae-43e3-aeec-52934b2be942") : secret "cluster-monitoring-operator-tls" not found Jul 05 00:13:18.573238 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:18.573195 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:18.618839 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:18.618812 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:19.991979 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:19.991932 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:19.992450 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:19.992079 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 00:13:19.992450 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:19.992141 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls podName:10b6eca4-7fae-43e3-aeec-52934b2be942 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:23.992127316 +0000 UTC m=+241.928492673 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tbvtg" (UID: "10b6eca4-7fae-43e3-aeec-52934b2be942") : secret "cluster-monitoring-operator-tls" not found Jul 05 00:13:20.242249 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.242173 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:20.342064 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.342028 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:20.906048 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.906020 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lw7zc"] Jul 05 00:13:20.907945 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.907927 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:20.912297 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.912280 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 05 00:13:20.912391 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.912357 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 05 00:13:20.912429 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.912392 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 05 00:13:20.912429 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.912357 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m5c25\"" Jul 05 00:13:20.914016 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.913997 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 05 00:13:20.937451 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.937430 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lw7zc"] Jul 05 00:13:20.983233 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.983204 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:20.999839 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.999816 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/37f1b5d2-c173-4744-a850-6053f1f3bd35-data-volume\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.000166 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.999854 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/37f1b5d2-c173-4744-a850-6053f1f3bd35-crio-socket\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.000166 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.999876 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.000166 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.999962 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hm2h\" (UniqueName: \"kubernetes.io/projected/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-api-access-4hm2h\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.000166 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:20.999996 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.092461 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:21.092436 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:21.100471 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:21.100444 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.100562 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:21.100497 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hm2h\" (UniqueName: \"kubernetes.io/projected/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-api-access-4hm2h\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.100562 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:21.100528 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.100665 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:21.100566 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 00:13:21.100665 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:21.100588 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/37f1b5d2-c173-4744-a850-6053f1f3bd35-data-volume\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.100665 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:21.100648 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/37f1b5d2-c173-4744-a850-6053f1f3bd35-crio-socket\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.100794 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:21.100672 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 00:13:21.100794 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:21.100705 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm podName:37f1b5d2-c173-4744-a850-6053f1f3bd35 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:21.600684995 +0000 UTC m=+239.537050356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lw7zc" (UID: "37f1b5d2-c173-4744-a850-6053f1f3bd35") : configmap "kube-rbac-proxy" not found Jul 05 00:13:21.100794 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:21.100740 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls podName:37f1b5d2-c173-4744-a850-6053f1f3bd35 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:21.6007257 +0000 UTC m=+239.537091056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lw7zc" (UID: "37f1b5d2-c173-4744-a850-6053f1f3bd35") : secret "insights-runtime-extractor-tls" not found Jul 05 00:13:21.100794 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:21.100772 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/37f1b5d2-c173-4744-a850-6053f1f3bd35-crio-socket\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.100982 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:21.100971 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/37f1b5d2-c173-4744-a850-6053f1f3bd35-data-volume\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.110826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:21.110807 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hm2h\" (UniqueName: \"kubernetes.io/projected/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-api-access-4hm2h\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.217698 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:21.217634 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:21.320366 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:21.320338 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:21.604474 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:21.604402 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.604474 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:21.604460 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:21.604647 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:21.604520 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 00:13:21.604647 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:21.604588 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 00:13:21.604647 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:21.604589 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm podName:37f1b5d2-c173-4744-a850-6053f1f3bd35 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:22.604573247 +0000 UTC m=+240.540938604 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lw7zc" (UID: "37f1b5d2-c173-4744-a850-6053f1f3bd35") : configmap "kube-rbac-proxy" not found Jul 05 00:13:21.604647 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:21.604647 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls podName:37f1b5d2-c173-4744-a850-6053f1f3bd35 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:22.604634943 +0000 UTC m=+240.541000299 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lw7zc" (UID: "37f1b5d2-c173-4744-a850-6053f1f3bd35") : secret "insights-runtime-extractor-tls" not found Jul 05 00:13:22.578892 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:22.578855 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:22.611979 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:22.611947 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:22.612126 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:22.612013 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:22.612126 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:22.612059 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 00:13:22.612220 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:22.612130 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm podName:37f1b5d2-c173-4744-a850-6053f1f3bd35 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:24.612107856 +0000 UTC m=+242.548473212 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lw7zc" (UID: "37f1b5d2-c173-4744-a850-6053f1f3bd35") : configmap "kube-rbac-proxy" not found Jul 05 00:13:22.612220 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:22.612169 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 00:13:22.612220 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:22.612212 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls podName:37f1b5d2-c173-4744-a850-6053f1f3bd35 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:24.612200536 +0000 UTC m=+242.548565896 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lw7zc" (UID: "37f1b5d2-c173-4744-a850-6053f1f3bd35") : secret "insights-runtime-extractor-tls" not found Jul 05 00:13:22.680069 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:22.680043 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:22.693614 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:22.693597 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9595p_215cde65-c249-45c8-b941-488c4ab6b568/dns-node-resolver/0.log" Jul 05 00:13:23.368877 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:23.368847 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:23.693322 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:23.693304 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8s5zw_f2fde0b3-a812-4baf-9da4-719e7b227fd9/node-ca/0.log" Jul 05 00:13:24.023720 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:24.023642 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:24.023856 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:24.023789 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 00:13:24.023894 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:24.023856 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls podName:10b6eca4-7fae-43e3-aeec-52934b2be942 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:32.023838164 +0000 UTC m=+249.960203528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tbvtg" (UID: "10b6eca4-7fae-43e3-aeec-52934b2be942") : secret "cluster-monitoring-operator-tls" not found Jul 05 00:13:24.493757 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:24.493728 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c9fwj_a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4/serve-healthcheck-canary/0.log" Jul 05 00:13:24.627550 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:24.627521 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:24.627733 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:24.627578 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:24.627733 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:24.627643 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 05 00:13:24.627733 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:24.627704 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 00:13:24.627733 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:24.627712 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm podName:37f1b5d2-c173-4744-a850-6053f1f3bd35 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:28.627696654 +0000 UTC m=+246.564062011 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lw7zc" (UID: "37f1b5d2-c173-4744-a850-6053f1f3bd35") : configmap "kube-rbac-proxy" not found Jul 05 00:13:24.627930 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:24.627760 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls podName:37f1b5d2-c173-4744-a850-6053f1f3bd35 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:28.627746469 +0000 UTC m=+246.564111834 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lw7zc" (UID: "37f1b5d2-c173-4744-a850-6053f1f3bd35") : secret "insights-runtime-extractor-tls" not found Jul 05 00:13:25.424558 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:25.424520 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:25.650158 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:25.650130 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:25.991436 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:25.991402 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:26.092782 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:26.092761 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:26.192414 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:26.192387 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:26.347153 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:26.347121 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:26.447091 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:26.447057 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:26.547189 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:26.547161 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:26.651235 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:26.651212 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:27.284951 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:27.284922 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:27.677607 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:27.677575 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:28.081996 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:28.081920 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:28.480247 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:28.480218 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:28.658992 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:28.658958 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:28.659147 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:28.659012 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:28.659147 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:28.659140 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 05 00:13:28.659222 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:28.659211 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls podName:37f1b5d2-c173-4744-a850-6053f1f3bd35 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:36.659192995 +0000 UTC m=+254.595558371 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lw7zc" (UID: "37f1b5d2-c173-4744-a850-6053f1f3bd35") : secret "insights-runtime-extractor-tls" not found Jul 05 00:13:28.659566 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:28.659549 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37f1b5d2-c173-4744-a850-6053f1f3bd35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:28.682914 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:28.682886 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:29.291533 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:29.291503 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:29.878198 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:29.878169 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:29.928346 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:29.928318 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:29.980676 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:29.980651 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:30.076968 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:30.076941 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:30.130514 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:30.130445 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:30.279215 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:30.279191 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:30.681098 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:30.681068 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:31.482252 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:31.482220 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:31.879373 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:31.879292 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:32.076918 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:32.076878 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:32.082947 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:32.082919 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:32.083067 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:32.083054 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 05 00:13:32.083120 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:13:32.083111 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls podName:10b6eca4-7fae-43e3-aeec-52934b2be942 nodeName:}" failed. No retries permitted until 2026-07-05 00:13:48.083097221 +0000 UTC m=+266.019462577 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tbvtg" (UID: "10b6eca4-7fae-43e3-aeec-52934b2be942") : secret "cluster-monitoring-operator-tls" not found Jul 05 00:13:32.283302 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:32.283273 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:32.477099 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:32.477065 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:32.687499 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:32.687463 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:33.685738 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:33.685697 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:33.885658 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:33.885629 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:34.080881 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:34.080802 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:34.277842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:34.277812 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:34.299986 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:34.299955 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:13:34.302319 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:34.302302 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8e8f787-142b-4f26-9a75-9cc3b6976f35-metrics-certs\") pod \"network-metrics-daemon-td7ll\" (UID: \"a8e8f787-142b-4f26-9a75-9cc3b6976f35\") " pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:13:34.477029 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:34.476997 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:34.522762 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:34.522729 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m9r6w\"" Jul 05 00:13:34.530822 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:34.530808 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-td7ll" Jul 05 00:13:34.652355 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:34.652328 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-td7ll"] Jul 05 00:13:34.655697 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:13:34.655671 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8e8f787_142b_4f26_9a75_9cc3b6976f35.slice/crio-dfb5b136914383546a7b78d2c8ad939cd548e56f0f161eea75ffacb5b809a440 WatchSource:0}: Error finding container dfb5b136914383546a7b78d2c8ad939cd548e56f0f161eea75ffacb5b809a440: Status 404 returned error can't find the container with id dfb5b136914383546a7b78d2c8ad939cd548e56f0f161eea75ffacb5b809a440 Jul 05 00:13:35.332325 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:35.332292 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:35.515766 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:35.515720 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-td7ll" event={"ID":"a8e8f787-142b-4f26-9a75-9cc3b6976f35","Type":"ContainerStarted","Data":"dfb5b136914383546a7b78d2c8ad939cd548e56f0f161eea75ffacb5b809a440"} Jul 05 00:13:35.880552 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:35.880470 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:36.086357 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:36.086326 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:36.280138 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:36.280096 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:36.519917 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:36.519880 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-td7ll" event={"ID":"a8e8f787-142b-4f26-9a75-9cc3b6976f35","Type":"ContainerStarted","Data":"fa7be5a47126f1bc7651be1c696c98348be7640b619b5e63edc82e14d96ca0b7"} Jul 05 00:13:36.520271 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:36.519948 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-td7ll" event={"ID":"a8e8f787-142b-4f26-9a75-9cc3b6976f35","Type":"ContainerStarted","Data":"1ab1c8995f41367e7d4a55a00cdf80483fb4e0c8fc9e2135067843c2bc363e9f"} Jul 05 00:13:36.534722 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:36.534641 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-td7ll" podStartSLOduration=253.623035444 podStartE2EDuration="4m14.534617218s" totalImagesPullingTime="911.581774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:13:36.53389382 +0000 UTC m=+254.470259200" watchObservedRunningTime="2026-07-05 00:13:36.534617218 +0000 UTC m=+254.470982597" Jul 05 00:13:36.683789 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:36.683759 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:36.719531 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:36.719506 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:36.721822 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:36.721798 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37f1b5d2-c173-4744-a850-6053f1f3bd35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw7zc\" (UID: \"37f1b5d2-c173-4744-a850-6053f1f3bd35\") " pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:36.819265 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:36.819199 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m5c25\"" Jul 05 00:13:36.827373 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:36.827357 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lw7zc" Jul 05 00:13:36.881808 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:36.881784 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:36.948563 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:36.948527 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lw7zc"] Jul 05 00:13:36.952317 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:13:36.952289 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f1b5d2_c173_4744_a850_6053f1f3bd35.slice/crio-1f1284614ac80dac4cd87ca8ab12991616290ecec6bff9e0a528a1184c52bd7e WatchSource:0}: Error finding container 1f1284614ac80dac4cd87ca8ab12991616290ecec6bff9e0a528a1184c52bd7e: Status 404 returned error can't find the container with id 1f1284614ac80dac4cd87ca8ab12991616290ecec6bff9e0a528a1184c52bd7e Jul 05 00:13:37.528994 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:37.528960 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lw7zc" event={"ID":"37f1b5d2-c173-4744-a850-6053f1f3bd35","Type":"ContainerStarted","Data":"1f1284614ac80dac4cd87ca8ab12991616290ecec6bff9e0a528a1184c52bd7e"} Jul 05 00:13:41.402967 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:41.402919 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:41.451492 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:41.451455 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:41.506347 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:41.506320 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:41.543838 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:41.543811 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lw7zc" event={"ID":"37f1b5d2-c173-4744-a850-6053f1f3bd35","Type":"ContainerStarted","Data":"3235b2623617797c95675a811c35ed09b37d74c9f1633cb4c42593a96bba36a8"} Jul 05 00:13:41.544000 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:41.543868 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lw7zc" event={"ID":"37f1b5d2-c173-4744-a850-6053f1f3bd35","Type":"ContainerStarted","Data":"e8e3c0c3ca77cda0865293488d8830331eead44a914fe99ddb859773ccb6a3e4"} Jul 05 00:13:41.704651 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:41.704516 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:42.124187 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.124097 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:42.127376 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.127345 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-49j5s"] Jul 05 00:13:42.129423 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.129350 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-49j5s" Jul 05 00:13:42.131977 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.131957 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-nlhxl\"" Jul 05 00:13:42.132278 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.132257 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 05 00:13:42.132380 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.132308 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 05 00:13:42.142705 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.142675 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-49j5s"] Jul 05 00:13:42.155454 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.155400 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5cb9dfb49c-cdph4"] Jul 05 00:13:42.157348 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.157329 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.160028 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.159997 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 05 00:13:42.160146 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.160026 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qmcqv\"" Jul 05 00:13:42.160146 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.160046 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 05 00:13:42.160146 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.160058 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 05 00:13:42.166384 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.166362 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 05 00:13:42.171918 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.171875 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cb9dfb49c-cdph4"] Jul 05 00:13:42.221395 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.221352 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:42.265015 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.264974 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/41415125-9abb-4915-832f-7b8dbf7fd381-installation-pull-secrets\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.265195 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.265082 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmbzd\" (UniqueName: \"kubernetes.io/projected/41415125-9abb-4915-832f-7b8dbf7fd381-kube-api-access-rmbzd\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.265195 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.265113 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41415125-9abb-4915-832f-7b8dbf7fd381-bound-sa-token\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.265195 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.265128 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkqns\" (UniqueName: \"kubernetes.io/projected/a8354f6b-5bb0-4e2d-ac7e-60c022ebca48-kube-api-access-rkqns\") pod \"downloads-685bfb5f64-49j5s\" (UID: \"a8354f6b-5bb0-4e2d-ac7e-60c022ebca48\") " pod="openshift-console/downloads-685bfb5f64-49j5s" Jul 05 00:13:42.265195 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.265155 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/41415125-9abb-4915-832f-7b8dbf7fd381-registry-certificates\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.265402 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.265236 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/41415125-9abb-4915-832f-7b8dbf7fd381-registry-tls\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.265402 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.265283 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/41415125-9abb-4915-832f-7b8dbf7fd381-trusted-ca\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.265402 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.265317 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/41415125-9abb-4915-832f-7b8dbf7fd381-image-registry-private-configuration\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.265402 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.265346 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/41415125-9abb-4915-832f-7b8dbf7fd381-ca-trust-extracted\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.323831 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.323797 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:42.366185 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.366153 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/41415125-9abb-4915-832f-7b8dbf7fd381-installation-pull-secrets\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.366373 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.366218 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmbzd\" (UniqueName: \"kubernetes.io/projected/41415125-9abb-4915-832f-7b8dbf7fd381-kube-api-access-rmbzd\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.366373 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.366247 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41415125-9abb-4915-832f-7b8dbf7fd381-bound-sa-token\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.366373 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.366264 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkqns\" (UniqueName: \"kubernetes.io/projected/a8354f6b-5bb0-4e2d-ac7e-60c022ebca48-kube-api-access-rkqns\") pod \"downloads-685bfb5f64-49j5s\" (UID: \"a8354f6b-5bb0-4e2d-ac7e-60c022ebca48\") " pod="openshift-console/downloads-685bfb5f64-49j5s" Jul 05 00:13:42.366373 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.366285 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/41415125-9abb-4915-832f-7b8dbf7fd381-registry-certificates\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.366373 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.366319 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/41415125-9abb-4915-832f-7b8dbf7fd381-registry-tls\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.366373 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.366351 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/41415125-9abb-4915-832f-7b8dbf7fd381-trusted-ca\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.366697 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.366378 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/41415125-9abb-4915-832f-7b8dbf7fd381-image-registry-private-configuration\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.366697 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.366403 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/41415125-9abb-4915-832f-7b8dbf7fd381-ca-trust-extracted\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.366877 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.366852 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/41415125-9abb-4915-832f-7b8dbf7fd381-ca-trust-extracted\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.367403 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.367378 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/41415125-9abb-4915-832f-7b8dbf7fd381-registry-certificates\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.367813 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.367791 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/41415125-9abb-4915-832f-7b8dbf7fd381-trusted-ca\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.369154 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.369133 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/41415125-9abb-4915-832f-7b8dbf7fd381-installation-pull-secrets\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.369505 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.369485 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/41415125-9abb-4915-832f-7b8dbf7fd381-registry-tls\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.369755 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.369733 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/41415125-9abb-4915-832f-7b8dbf7fd381-image-registry-private-configuration\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.375541 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.375487 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41415125-9abb-4915-832f-7b8dbf7fd381-bound-sa-token\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.375954 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.375930 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmbzd\" (UniqueName: \"kubernetes.io/projected/41415125-9abb-4915-832f-7b8dbf7fd381-kube-api-access-rmbzd\") pod \"image-registry-5cb9dfb49c-cdph4\" (UID: \"41415125-9abb-4915-832f-7b8dbf7fd381\") " pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.376373 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.376351 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkqns\" (UniqueName: \"kubernetes.io/projected/a8354f6b-5bb0-4e2d-ac7e-60c022ebca48-kube-api-access-rkqns\") pod \"downloads-685bfb5f64-49j5s\" (UID: \"a8354f6b-5bb0-4e2d-ac7e-60c022ebca48\") " pod="openshift-console/downloads-685bfb5f64-49j5s" Jul 05 00:13:42.421736 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.421689 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:42.439698 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.439673 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-49j5s" Jul 05 00:13:42.468457 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.468432 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:42.521177 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.521129 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:42.591950 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.591897 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-49j5s"] Jul 05 00:13:42.594378 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.594355 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:42.622758 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.622737 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cb9dfb49c-cdph4"] Jul 05 00:13:42.645578 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:42.645549 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:43.293800 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:43.293761 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:43.491715 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:43.491683 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:43.591670 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:43.591603 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:43.604294 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:13:43.604261 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8354f6b_5bb0_4e2d_ac7e_60c022ebca48.slice/crio-ab911fa3c19672f6c7b83fa871a1cedf1a8a1521969aaea6329382d353f9a3d4 WatchSource:0}: Error finding container ab911fa3c19672f6c7b83fa871a1cedf1a8a1521969aaea6329382d353f9a3d4: Status 404 returned error can't find the container with id ab911fa3c19672f6c7b83fa871a1cedf1a8a1521969aaea6329382d353f9a3d4 Jul 05 00:13:43.604656 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:13:43.604642 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41415125_9abb_4915_832f_7b8dbf7fd381.slice/crio-29ea0ecaf3b7dc7a517941c73812137b45c4cad35fc66af4d913caab72b23a86 WatchSource:0}: Error finding container 29ea0ecaf3b7dc7a517941c73812137b45c4cad35fc66af4d913caab72b23a86: Status 404 returned error can't find the container with id 29ea0ecaf3b7dc7a517941c73812137b45c4cad35fc66af4d913caab72b23a86 Jul 05 00:13:44.511856 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:44.511816 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:44.557298 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:44.557260 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lw7zc" event={"ID":"37f1b5d2-c173-4744-a850-6053f1f3bd35","Type":"ContainerStarted","Data":"8d1d41858abd99b17369784b719192e5920b7f2c5f96b7584b360afb6a95a136"} Jul 05 00:13:44.558932 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:44.558820 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" event={"ID":"41415125-9abb-4915-832f-7b8dbf7fd381","Type":"ContainerStarted","Data":"7ecf67cac34b8f2f87569cfdc0727cd90ec8010eba5cb41becbb3688c43df101"} Jul 05 00:13:44.558932 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:44.558866 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" event={"ID":"41415125-9abb-4915-832f-7b8dbf7fd381","Type":"ContainerStarted","Data":"29ea0ecaf3b7dc7a517941c73812137b45c4cad35fc66af4d913caab72b23a86"} Jul 05 00:13:44.558932 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:44.558893 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:13:44.560159 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:44.560124 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-49j5s" event={"ID":"a8354f6b-5bb0-4e2d-ac7e-60c022ebca48","Type":"ContainerStarted","Data":"ab911fa3c19672f6c7b83fa871a1cedf1a8a1521969aaea6329382d353f9a3d4"} Jul 05 00:13:44.574766 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:44.574713 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lw7zc" podStartSLOduration=17.988534299 podStartE2EDuration="24.574699116s" totalImagesPullingTime="6.586164817s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:13:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:13:44.573535333 +0000 UTC m=+262.509900725" watchObservedRunningTime="2026-07-05 00:13:44.574699116 +0000 UTC m=+262.511064494" Jul 05 00:13:44.594837 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:44.594777 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" podStartSLOduration=2.5947575929999998 podStartE2EDuration="2.594757593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:13:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:13:44.593740862 +0000 UTC m=+262.530106270" watchObservedRunningTime="2026-07-05 00:13:44.594757593 +0000 UTC m=+262.531122975" Jul 05 00:13:45.692175 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:45.692132 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:48.118678 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:48.118632 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:48.121511 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:48.121482 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10b6eca4-7fae-43e3-aeec-52934b2be942-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tbvtg\" (UID: \"10b6eca4-7fae-43e3-aeec-52934b2be942\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:48.419864 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:48.419809 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mzjx4\"" Jul 05 00:13:48.427274 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:48.427238 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" Jul 05 00:13:48.565368 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:48.565324 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg"] Jul 05 00:13:48.568875 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:13:48.568850 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10b6eca4_7fae_43e3_aeec_52934b2be942.slice/crio-d3ec9fc91ec218494fcf4fe0651be86fbaf3fea3bf179d31bc7b043a01ad2b9f WatchSource:0}: Error finding container d3ec9fc91ec218494fcf4fe0651be86fbaf3fea3bf179d31bc7b043a01ad2b9f: Status 404 returned error can't find the container with id d3ec9fc91ec218494fcf4fe0651be86fbaf3fea3bf179d31bc7b043a01ad2b9f Jul 05 00:13:48.577099 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:48.577070 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" event={"ID":"10b6eca4-7fae-43e3-aeec-52934b2be942","Type":"ContainerStarted","Data":"d3ec9fc91ec218494fcf4fe0651be86fbaf3fea3bf179d31bc7b043a01ad2b9f"} Jul 05 00:13:49.311594 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:49.311558 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:50.857086 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:50.856990 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:50.961118 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:50.961025 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:51.026548 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:51.026497 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:51.185577 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:51.185545 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:51.591102 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:51.591001 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" event={"ID":"10b6eca4-7fae-43e3-aeec-52934b2be942","Type":"ContainerStarted","Data":"0d9a5f212907dfb7d228bf1c08123f348945af5e6e17a8d688bd24440f8ee3f1"} Jul 05 00:13:51.607082 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:51.607030 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tbvtg" podStartSLOduration=33.644143107 podStartE2EDuration="35.607011542s" totalImagesPullingTime="1.962868435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:13:51.606067633 +0000 UTC m=+269.542433011" watchObservedRunningTime="2026-07-05 00:13:51.607011542 +0000 UTC m=+269.543376983" Jul 05 00:13:51.709404 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:51.709370 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:51.764224 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:51.764191 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:55.025895 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:55.025864 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:55.087356 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:55.087312 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:55.374652 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:55.374569 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:55.984376 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:55.984340 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:56.037942 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:56.037888 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:56.692137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:56.692105 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:57.728127 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:57.728092 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:57.793124 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:57.793086 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:57.844034 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:57.843999 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:59.309352 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:59.309307 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:13:59.365379 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:13:59.365341 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:00.160402 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.160371 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:00.360795 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.360769 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:00.394674 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.394646 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5db2k"] Jul 05 00:14:00.416132 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.416080 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.418515 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.418488 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qgkjb\"" Jul 05 00:14:00.418640 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.418530 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 05 00:14:00.418744 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.418711 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 05 00:14:00.419865 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.419847 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 05 00:14:00.419988 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.419847 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 05 00:14:00.444382 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.444349 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:00.525506 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.525479 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bchmm\" (UniqueName: \"kubernetes.io/projected/bebc5085-24b5-4c18-bef7-aabcb2944337-kube-api-access-bchmm\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.525648 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.525539 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bebc5085-24b5-4c18-bef7-aabcb2944337-sys\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.525648 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.525623 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-wtmp\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.525731 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.525668 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bebc5085-24b5-4c18-bef7-aabcb2944337-metrics-client-ca\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.525731 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.525710 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-accelerators-collector-config\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.525808 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.525735 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-tls\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.525808 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.525758 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.525808 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.525801 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bebc5085-24b5-4c18-bef7-aabcb2944337-root\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.525895 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.525822 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-textfile\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.626205 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626177 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-wtmp\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.626364 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626216 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bebc5085-24b5-4c18-bef7-aabcb2944337-metrics-client-ca\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.626364 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626234 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-accelerators-collector-config\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.626364 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626294 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-tls\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.626364 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626330 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.626364 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626346 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-wtmp\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.626364 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626356 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bebc5085-24b5-4c18-bef7-aabcb2944337-root\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.626632 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626393 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bebc5085-24b5-4c18-bef7-aabcb2944337-root\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.626632 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626395 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-textfile\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.626632 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626438 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bchmm\" (UniqueName: \"kubernetes.io/projected/bebc5085-24b5-4c18-bef7-aabcb2944337-kube-api-access-bchmm\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.626632 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626495 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bebc5085-24b5-4c18-bef7-aabcb2944337-sys\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.626632 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:14:00.626543 2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 05 00:14:00.626632 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626563 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bebc5085-24b5-4c18-bef7-aabcb2944337-sys\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.626632 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:14:00.626609 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-tls podName:bebc5085-24b5-4c18-bef7-aabcb2944337 nodeName:}" failed. No retries permitted until 2026-07-05 00:14:01.126589811 +0000 UTC m=+279.062955174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-tls") pod "node-exporter-5db2k" (UID: "bebc5085-24b5-4c18-bef7-aabcb2944337") : secret "node-exporter-tls" not found Jul 05 00:14:00.627036 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626704 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-textfile\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.627036 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626825 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-accelerators-collector-config\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.627036 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.626957 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bebc5085-24b5-4c18-bef7-aabcb2944337-metrics-client-ca\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.628758 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.628737 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.637066 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.637048 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bchmm\" (UniqueName: \"kubernetes.io/projected/bebc5085-24b5-4c18-bef7-aabcb2944337-kube-api-access-bchmm\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:00.756359 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.756298 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:00.856764 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.856740 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:00.957064 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:00.957033 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:01.057478 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.057402 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:01.130248 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.130212 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-tls\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:01.133218 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.133181 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bebc5085-24b5-4c18-bef7-aabcb2944337-node-exporter-tls\") pod \"node-exporter-5db2k\" (UID: \"bebc5085-24b5-4c18-bef7-aabcb2944337\") " pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:01.324405 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.324322 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5db2k" Jul 05 00:14:01.336530 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:14:01.336496 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbebc5085_24b5_4c18_bef7_aabcb2944337.slice/crio-8de67c34219cd91517637318a5a60c7ff2eba5ea121169ce74df1c277fae7d80 WatchSource:0}: Error finding container 8de67c34219cd91517637318a5a60c7ff2eba5ea121169ce74df1c277fae7d80: Status 404 returned error can't find the container with id 8de67c34219cd91517637318a5a60c7ff2eba5ea121169ce74df1c277fae7d80 Jul 05 00:14:01.383166 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.383131 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:01.470090 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.470059 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 00:14:01.474109 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.474080 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.475476 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.475451 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:01.480138 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.480118 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 05 00:14:01.480238 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.480210 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 05 00:14:01.480471 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.480450 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 05 00:14:01.480766 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.480747 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 05 00:14:01.480857 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.480777 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 05 00:14:01.480935 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.480781 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 05 00:14:01.481025 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.481006 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 05 00:14:01.481138 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.481066 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 05 00:14:01.481550 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.481416 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fkhcm\"" Jul 05 00:14:01.481550 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.481452 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 05 00:14:01.490820 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.490796 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 00:14:01.584137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.584020 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:01.624585 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.624547 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-49j5s" event={"ID":"a8354f6b-5bb0-4e2d-ac7e-60c022ebca48","Type":"ContainerStarted","Data":"2c29fcd0e2eba042d30c19c819276e54c59fe3945adbe3cdd38ad46bf9bfdeef"} Jul 05 00:14:01.625326 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.624851 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-49j5s" Jul 05 00:14:01.626185 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.626127 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5db2k" event={"ID":"bebc5085-24b5-4c18-bef7-aabcb2944337","Type":"ContainerStarted","Data":"8de67c34219cd91517637318a5a60c7ff2eba5ea121169ce74df1c277fae7d80"} Jul 05 00:14:01.634817 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.634792 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57rcn\" (UniqueName: \"kubernetes.io/projected/99a275a4-2248-4071-90dd-5c36deab90f6-kube-api-access-57rcn\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.634981 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.634829 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a275a4-2248-4071-90dd-5c36deab90f6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.634981 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.634863 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/99a275a4-2248-4071-90dd-5c36deab90f6-config-out\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.634981 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.634887 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.634981 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.634932 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.634981 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.634959 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/99a275a4-2248-4071-90dd-5c36deab90f6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.635265 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.634983 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-web-config\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.635265 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.635012 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.635265 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.635081 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.635265 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.635116 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-config-volume\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.635265 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.635181 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/99a275a4-2248-4071-90dd-5c36deab90f6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.635265 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.635236 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.635565 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.635281 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99a275a4-2248-4071-90dd-5c36deab90f6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.637346 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.637323 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-49j5s" Jul 05 00:14:01.640938 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.640877 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-49j5s" podStartSLOduration=2.460662668 podStartE2EDuration="19.640860134s" totalImagesPullingTime="17.180197466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:13:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:14:01.640013894 +0000 UTC m=+279.576379272" watchObservedRunningTime="2026-07-05 00:14:01.640860134 +0000 UTC m=+279.577225513" Jul 05 00:14:01.736561 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.736525 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/99a275a4-2248-4071-90dd-5c36deab90f6-config-out\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.736561 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.736567 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.736827 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.736596 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.736827 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.736626 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/99a275a4-2248-4071-90dd-5c36deab90f6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.736827 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.736650 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-web-config\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.736827 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.736678 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.736827 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.736717 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.736827 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.736751 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-config-volume\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.736827 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.736805 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/99a275a4-2248-4071-90dd-5c36deab90f6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.737162 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.736832 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.737162 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.736866 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99a275a4-2248-4071-90dd-5c36deab90f6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.737162 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.736931 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57rcn\" (UniqueName: \"kubernetes.io/projected/99a275a4-2248-4071-90dd-5c36deab90f6-kube-api-access-57rcn\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.737162 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.736980 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a275a4-2248-4071-90dd-5c36deab90f6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.737993 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.737961 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a275a4-2248-4071-90dd-5c36deab90f6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.740723 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.739282 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99a275a4-2248-4071-90dd-5c36deab90f6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.741354 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.741329 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.741569 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.741544 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/99a275a4-2248-4071-90dd-5c36deab90f6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.742826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.742682 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.745605 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.745476 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.745605 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.745493 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.745605 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.745540 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-config-volume\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.746805 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.745788 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/99a275a4-2248-4071-90dd-5c36deab90f6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.747166 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.747143 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57rcn\" (UniqueName: \"kubernetes.io/projected/99a275a4-2248-4071-90dd-5c36deab90f6-kube-api-access-57rcn\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.747766 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.747744 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.747967 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.747887 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/99a275a4-2248-4071-90dd-5c36deab90f6-web-config\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.752470 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.752448 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/99a275a4-2248-4071-90dd-5c36deab90f6-config-out\") pod \"alertmanager-main-0\" (UID: \"99a275a4-2248-4071-90dd-5c36deab90f6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.787229 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.787179 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 05 00:14:01.967987 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:01.967860 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 05 00:14:01.973397 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:14:01.973352 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a275a4_2248_4071_90dd_5c36deab90f6.slice/crio-3028ea9f42575c67aadd1176c589a480e8a3f7f46faaa634b5205f7502847344 WatchSource:0}: Error finding container 3028ea9f42575c67aadd1176c589a480e8a3f7f46faaa634b5205f7502847344: Status 404 returned error can't find the container with id 3028ea9f42575c67aadd1176c589a480e8a3f7f46faaa634b5205f7502847344 Jul 05 00:14:02.340195 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:02.340161 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:02.432937 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:02.432818 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:02.633644 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:02.632483 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5db2k" event={"ID":"bebc5085-24b5-4c18-bef7-aabcb2944337","Type":"ContainerStarted","Data":"c6955b09d1acb5ca1eed584e12c51952173042fa72a7a1263436028d20b7e90b"} Jul 05 00:14:02.634008 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:02.633981 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99a275a4-2248-4071-90dd-5c36deab90f6","Type":"ContainerStarted","Data":"3028ea9f42575c67aadd1176c589a480e8a3f7f46faaa634b5205f7502847344"} Jul 05 00:14:03.172166 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:03.172098 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:03.246086 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:03.245984 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:03.308566 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:03.308533 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:03.380369 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:03.380341 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:03.456363 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:03.456278 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:03.644889 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:03.644775 2558 generic.go:356] "Generic (PLEG): container finished" podID="bebc5085-24b5-4c18-bef7-aabcb2944337" containerID="c6955b09d1acb5ca1eed584e12c51952173042fa72a7a1263436028d20b7e90b" exitCode=0 Jul 05 00:14:03.644889 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:03.644871 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5db2k" event={"ID":"bebc5085-24b5-4c18-bef7-aabcb2944337","Type":"ContainerDied","Data":"c6955b09d1acb5ca1eed584e12c51952173042fa72a7a1263436028d20b7e90b"} Jul 05 00:14:03.985124 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:03.985094 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:04.185170 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:04.185131 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:04.259921 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:14:04.259871 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-s764k" podUID="dd7a485d-b0d9-4e77-bf59-bdf7ed11f365" Jul 05 00:14:04.651085 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:04.651047 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5db2k" event={"ID":"bebc5085-24b5-4c18-bef7-aabcb2944337","Type":"ContainerStarted","Data":"d94c47ec920ebce420685e9f66c3255f3b814e5f50e94e5664f46998d97c7e13"} Jul 05 00:14:04.651550 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:04.651106 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5db2k" event={"ID":"bebc5085-24b5-4c18-bef7-aabcb2944337","Type":"ContainerStarted","Data":"2bc265126c872058154e98bc03cd4104e97a42db23f4bb92e9c0830113df4f9c"} Jul 05 00:14:04.652848 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:04.652818 2558 generic.go:356] "Generic (PLEG): container finished" podID="99a275a4-2248-4071-90dd-5c36deab90f6" containerID="fa15b1e30aa760fb8e359b6ddff7bf1e31edf8b2fc450a087b6b61218fa90bd1" exitCode=0 Jul 05 00:14:04.652990 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:04.652895 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s764k" Jul 05 00:14:04.652990 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:04.652922 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99a275a4-2248-4071-90dd-5c36deab90f6","Type":"ContainerDied","Data":"fa15b1e30aa760fb8e359b6ddff7bf1e31edf8b2fc450a087b6b61218fa90bd1"} Jul 05 00:14:04.671406 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:04.671362 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5db2k" podStartSLOduration=3.414219674 podStartE2EDuration="4.671348565s" totalImagesPullingTime="1.172085169s" totalInitContainerRuntime="85.043722ms" isStatefulPod=true podCreationTimestamp="2026-07-05 00:14:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:14:04.669168794 +0000 UTC m=+282.605534185" watchObservedRunningTime="2026-07-05 00:14:04.671348565 +0000 UTC m=+282.607713944" Jul 05 00:14:04.777304 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:04.777262 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:04.878274 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:04.878237 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:05.171376 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:05.171349 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:05.259722 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:05.259690 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:05.571799 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:05.571713 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5cb9dfb49c-cdph4" Jul 05 00:14:05.690656 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:05.690619 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:05.794199 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:05.794164 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:05.898222 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:05.898192 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:06.621732 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:06.621696 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:06.969744 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:06.969718 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:07.071287 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:07.071173 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:07.143711 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:07.143682 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:07.386661 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:07.386637 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:14:07.389283 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:07.389263 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd7a485d-b0d9-4e77-bf59-bdf7ed11f365-metrics-tls\") pod \"dns-default-s764k\" (UID: \"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365\") " pod="openshift-dns/dns-default-s764k" Jul 05 00:14:07.656007 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:07.655978 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-z9vd9\"" Jul 05 00:14:07.663551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:07.663529 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s764k" Jul 05 00:14:07.669354 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:07.669325 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99a275a4-2248-4071-90dd-5c36deab90f6","Type":"ContainerStarted","Data":"e3deca59f33ec8874f14b4136a380a89a10b3f1f5b6a51284593bab29578000c"} Jul 05 00:14:07.669461 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:07.669399 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99a275a4-2248-4071-90dd-5c36deab90f6","Type":"ContainerStarted","Data":"510e21841afd85617ca5f4f3fd181000c914e28dd8f4ec54214c56d33525af76"} Jul 05 00:14:07.669461 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:07.669440 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99a275a4-2248-4071-90dd-5c36deab90f6","Type":"ContainerStarted","Data":"04a5c165f87cab86e90463129a8c788a2639af0387cff792393475b82cb35045"} Jul 05 00:14:07.669566 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:07.669463 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99a275a4-2248-4071-90dd-5c36deab90f6","Type":"ContainerStarted","Data":"f65d73432651622868d98a6d3129a9a6f718c9f22e11aa78ec2c81f9c0f80bf7"} Jul 05 00:14:07.669566 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:07.669485 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99a275a4-2248-4071-90dd-5c36deab90f6","Type":"ContainerStarted","Data":"d9be8f1d95a6c0f21f04a4695b7f9f5b96fe105e6c848967d7dbe0f60837eb4b"} Jul 05 00:14:07.824861 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:07.824824 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-s764k"] Jul 05 00:14:07.828301 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:14:07.828272 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd7a485d_b0d9_4e77_bf59_bdf7ed11f365.slice/crio-607e5fad0dc7df1d1d0606b23a51d60939c5033ca2434db873bf3f12b26cb871 WatchSource:0}: Error finding container 607e5fad0dc7df1d1d0606b23a51d60939c5033ca2434db873bf3f12b26cb871: Status 404 returned error can't find the container with id 607e5fad0dc7df1d1d0606b23a51d60939c5033ca2434db873bf3f12b26cb871 Jul 05 00:14:08.675239 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:08.675203 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s764k" event={"ID":"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365","Type":"ContainerStarted","Data":"607e5fad0dc7df1d1d0606b23a51d60939c5033ca2434db873bf3f12b26cb871"} Jul 05 00:14:09.682797 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:09.682754 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"99a275a4-2248-4071-90dd-5c36deab90f6","Type":"ContainerStarted","Data":"8754f218023350f24a3d8c0ff6c360b75dda56be57a63d6a1ee1d4676b51f51b"} Jul 05 00:14:09.710136 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:09.710087 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.366492474 podStartE2EDuration="8.710066697s" totalImagesPullingTime="5.321125159s" totalInitContainerRuntime="22.449064ms" isStatefulPod=false podCreationTimestamp="2026-07-05 00:14:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:14:09.708612431 +0000 UTC m=+287.644977808" watchObservedRunningTime="2026-07-05 00:14:09.710066697 +0000 UTC m=+287.646432073" Jul 05 00:14:10.687552 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:10.687519 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s764k" event={"ID":"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365","Type":"ContainerStarted","Data":"a0d622e1c87900bb7ec2d4f6bae7ab6d75aceed278f731e2a6c93bb223dc62c3"} Jul 05 00:14:10.688043 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:10.687568 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s764k" event={"ID":"dd7a485d-b0d9-4e77-bf59-bdf7ed11f365","Type":"ContainerStarted","Data":"02a31fed4ae52b18b3e45ebb52f3c4d086d3765050ed755f10f1d6c67935f45b"} Jul 05 00:14:10.688043 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:10.687937 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-s764k" Jul 05 00:14:10.709604 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:10.709563 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-s764k" podStartSLOduration=251.449198917 podStartE2EDuration="4m13.709548221s" totalImagesPullingTime="2.260349304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:09:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:14:10.708338718 +0000 UTC m=+288.644704107" watchObservedRunningTime="2026-07-05 00:14:10.709548221 +0000 UTC m=+288.645913598" Jul 05 00:14:10.991724 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:10.991634 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:11.041344 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:11.041312 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:17.900523 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:17.900487 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:18.001078 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:18.001051 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:18.052231 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:18.052203 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:19.107292 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:19.107260 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:19.162054 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:19.162025 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:19.300656 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:19.300626 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:21.696038 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:21.696010 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-s764k" Jul 05 00:14:22.490467 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:22.490433 2558 kubelet.go:1648] "Image garbage collection succeeded" Jul 05 00:14:25.992215 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:25.992153 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:26.041152 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:26.041128 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:30.802920 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:30.802874 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:30.909223 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:30.909177 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:40.924624 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:40.924591 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:41.025583 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:41.025554 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:41.074790 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:41.074745 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:44.779436 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:44.779386 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" podUID="528a0a1e-b618-4606-a7ae-c78c01c8232b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 05 00:14:54.776477 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:54.776441 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" podUID="528a0a1e-b618-4606-a7ae-c78c01c8232b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 05 00:14:58.635963 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:58.635932 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:14:58.680770 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:14:58.680742 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:04.776042 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:04.776005 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" podUID="528a0a1e-b618-4606-a7ae-c78c01c8232b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 05 00:15:04.776380 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:04.776070 2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" Jul 05 00:15:04.776496 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:04.776468 2558 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"aeb68deee4f3ad7f2643f1321e0fb920e39c1c2417439a3e1da0d55ecf21a45d"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 05 00:15:04.776535 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:04.776516 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" podUID="528a0a1e-b618-4606-a7ae-c78c01c8232b" containerName="service-proxy" containerID="cri-o://aeb68deee4f3ad7f2643f1321e0fb920e39c1c2417439a3e1da0d55ecf21a45d" gracePeriod=30 Jul 05 00:15:05.873710 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:05.873682 2558 generic.go:356] "Generic (PLEG): container finished" podID="528a0a1e-b618-4606-a7ae-c78c01c8232b" containerID="aeb68deee4f3ad7f2643f1321e0fb920e39c1c2417439a3e1da0d55ecf21a45d" exitCode=2 Jul 05 00:15:05.873710 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:05.873718 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" event={"ID":"528a0a1e-b618-4606-a7ae-c78c01c8232b","Type":"ContainerDied","Data":"aeb68deee4f3ad7f2643f1321e0fb920e39c1c2417439a3e1da0d55ecf21a45d"} Jul 05 00:15:05.874130 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:05.873752 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d8457b789-zx6wj" event={"ID":"528a0a1e-b618-4606-a7ae-c78c01c8232b","Type":"ContainerStarted","Data":"be60b2aa2482cf17aecd58e587e9883a08a8190c7d8438ec4d7afe921d9aa058"} Jul 05 00:15:06.074362 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:06.074330 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:06.125690 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:06.125640 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:19.626759 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:19.626664 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:20.065497 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:20.065465 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:24.700610 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.700579 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5795879b4-lwbd7"] Jul 05 00:15:24.702777 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.702761 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.705625 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.705602 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 05 00:15:24.705753 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.705602 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 05 00:15:24.705753 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.705690 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 05 00:15:24.705915 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.705888 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 05 00:15:24.705989 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.705975 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 05 00:15:24.706036 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.706015 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-xbllt\"" Jul 05 00:15:24.710755 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.710710 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 05 00:15:24.714882 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.714864 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5795879b4-lwbd7"] Jul 05 00:15:24.834344 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.834316 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b8dq\" (UniqueName: \"kubernetes.io/projected/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-kube-api-access-8b8dq\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.834544 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.834366 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-telemeter-client-tls\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.834544 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.834413 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-metrics-client-ca\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.834544 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.834438 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-serving-certs-ca-bundle\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.834544 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.834460 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-secret-telemeter-client\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.834544 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.834483 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.834766 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.834565 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.834766 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.834594 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-federate-client-tls\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.935246 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.935215 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.935403 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.935254 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-federate-client-tls\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.935403 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.935284 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b8dq\" (UniqueName: \"kubernetes.io/projected/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-kube-api-access-8b8dq\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.935403 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.935309 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-telemeter-client-tls\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.935403 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.935341 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-metrics-client-ca\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.935403 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.935367 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-serving-certs-ca-bundle\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.935403 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.935389 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-secret-telemeter-client\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.935711 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.935412 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.936453 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.936428 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-serving-certs-ca-bundle\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.936586 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.936427 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.936586 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.936431 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-metrics-client-ca\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.938294 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.938267 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.938410 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.938355 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-secret-telemeter-client\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.938468 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.938452 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-telemeter-client-tls\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.938560 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.938543 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-federate-client-tls\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:24.942662 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:24.942641 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b8dq\" (UniqueName: \"kubernetes.io/projected/edc11376-16e8-4b39-a8ac-ab9da1ab56a4-kube-api-access-8b8dq\") pod \"telemeter-client-5795879b4-lwbd7\" (UID: \"edc11376-16e8-4b39-a8ac-ab9da1ab56a4\") " pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:25.012794 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:25.012742 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" Jul 05 00:15:25.141976 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:25.141863 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5795879b4-lwbd7"] Jul 05 00:15:25.144627 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:15:25.144601 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedc11376_16e8_4b39_a8ac_ab9da1ab56a4.slice/crio-9cde55d950cdefa80134cb576dfd63e1bdf320d6f6bb29e92ad8c34f038af5cb WatchSource:0}: Error finding container 9cde55d950cdefa80134cb576dfd63e1bdf320d6f6bb29e92ad8c34f038af5cb: Status 404 returned error can't find the container with id 9cde55d950cdefa80134cb576dfd63e1bdf320d6f6bb29e92ad8c34f038af5cb Jul 05 00:15:25.146513 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:25.146498 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 00:15:25.943755 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:25.943721 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" event={"ID":"edc11376-16e8-4b39-a8ac-ab9da1ab56a4","Type":"ContainerStarted","Data":"9cde55d950cdefa80134cb576dfd63e1bdf320d6f6bb29e92ad8c34f038af5cb"} Jul 05 00:15:26.622461 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:26.622394 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:26.676616 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:26.676587 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:27.953851 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:27.953809 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" event={"ID":"edc11376-16e8-4b39-a8ac-ab9da1ab56a4","Type":"ContainerStarted","Data":"9a22033160fd65d8bf141abe68493f4bc0db28c1a312c22538be1d8dcb9d54b6"} Jul 05 00:15:27.954237 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:27.953925 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" event={"ID":"edc11376-16e8-4b39-a8ac-ab9da1ab56a4","Type":"ContainerStarted","Data":"9396204e9b91aba0ea2438c4a96d0961c6f28f5617cec86ffaa494852d4da2f9"} Jul 05 00:15:27.954237 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:27.953955 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" event={"ID":"edc11376-16e8-4b39-a8ac-ab9da1ab56a4","Type":"ContainerStarted","Data":"2713d55629ec2ccb46534c86d48081b34e325fe2001e5938f1d157d5f6c9142c"} Jul 05 00:15:27.974784 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:27.974732 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5795879b4-lwbd7" podStartSLOduration=1.816742539 podStartE2EDuration="3.974698319s" totalImagesPullingTime="2.15795578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:15:27.973241216 +0000 UTC m=+365.909606619" watchObservedRunningTime="2026-07-05 00:15:27.974698319 +0000 UTC m=+365.911063698" Jul 05 00:15:28.751041 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:28.751008 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:28.853190 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:28.853156 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:34.769966 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:34.769897 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:34.821273 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:34.821238 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:46.718150 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:46.718111 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:46.771329 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:46.771300 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:49.285573 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:49.285537 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:49.334849 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:49.334816 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:49.385770 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:49.385741 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:55.994870 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:55.994836 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:15:56.095356 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:15:56.095325 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:16:01.987682 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:16:01.987646 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:16:02.035311 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:16:02.035280 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:09.433083 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:09.433007 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:09.527595 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:09.527561 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:35.400308 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:35.400274 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:35.453425 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:35.453390 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:40.984245 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:40.984207 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:41.763289 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:41.763256 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:45.001019 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:45.000984 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:45.101252 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:45.101219 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:45.997663 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:45.997631 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:46.103361 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:46.103331 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:46.250847 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:46.250769 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:46.362044 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:46.362015 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:48.654833 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:48.654797 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:49.041640 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:49.041571 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:51.413095 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.413062 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-w4fvb"] Jul 05 00:17:51.416004 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.415985 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w4fvb" Jul 05 00:17:51.418509 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.418493 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 05 00:17:51.419653 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.419621 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 05 00:17:51.419653 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.419644 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-tbt82\"" Jul 05 00:17:51.425610 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.425591 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-w4fvb"] Jul 05 00:17:51.464667 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.464640 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-794cn\" (UniqueName: \"kubernetes.io/projected/63a915f3-c50f-4a38-ba81-9566b267b1c0-kube-api-access-794cn\") pod \"cert-manager-cainjector-559cdd8c87-w4fvb\" (UID: \"63a915f3-c50f-4a38-ba81-9566b267b1c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w4fvb" Jul 05 00:17:51.464798 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.464709 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63a915f3-c50f-4a38-ba81-9566b267b1c0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-w4fvb\" (UID: \"63a915f3-c50f-4a38-ba81-9566b267b1c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w4fvb" Jul 05 00:17:51.565604 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.565573 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-794cn\" (UniqueName: \"kubernetes.io/projected/63a915f3-c50f-4a38-ba81-9566b267b1c0-kube-api-access-794cn\") pod \"cert-manager-cainjector-559cdd8c87-w4fvb\" (UID: \"63a915f3-c50f-4a38-ba81-9566b267b1c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w4fvb" Jul 05 00:17:51.565762 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.565645 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63a915f3-c50f-4a38-ba81-9566b267b1c0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-w4fvb\" (UID: \"63a915f3-c50f-4a38-ba81-9566b267b1c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w4fvb" Jul 05 00:17:51.573439 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.573406 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63a915f3-c50f-4a38-ba81-9566b267b1c0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-w4fvb\" (UID: \"63a915f3-c50f-4a38-ba81-9566b267b1c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w4fvb" Jul 05 00:17:51.573551 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.573489 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-794cn\" (UniqueName: \"kubernetes.io/projected/63a915f3-c50f-4a38-ba81-9566b267b1c0-kube-api-access-794cn\") pod \"cert-manager-cainjector-559cdd8c87-w4fvb\" (UID: \"63a915f3-c50f-4a38-ba81-9566b267b1c0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-w4fvb" Jul 05 00:17:51.725460 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.725371 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w4fvb" Jul 05 00:17:51.788412 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.788294 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:51.839500 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.839453 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:51.852141 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:51.852115 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-w4fvb"] Jul 05 00:17:51.855049 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:17:51.855021 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63a915f3_c50f_4a38_ba81_9566b267b1c0.slice/crio-6688b8f8cce561b74adb1c80c9d1ff275c11aa2327c8f6f5a1470fd3b049c2bc WatchSource:0}: Error finding container 6688b8f8cce561b74adb1c80c9d1ff275c11aa2327c8f6f5a1470fd3b049c2bc: Status 404 returned error can't find the container with id 6688b8f8cce561b74adb1c80c9d1ff275c11aa2327c8f6f5a1470fd3b049c2bc Jul 05 00:17:52.471687 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:52.471651 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w4fvb" event={"ID":"63a915f3-c50f-4a38-ba81-9566b267b1c0","Type":"ContainerStarted","Data":"6688b8f8cce561b74adb1c80c9d1ff275c11aa2327c8f6f5a1470fd3b049c2bc"} Jul 05 00:17:55.420503 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:55.420460 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:55.476089 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:55.476058 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:17:55.485211 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:55.485182 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w4fvb" event={"ID":"63a915f3-c50f-4a38-ba81-9566b267b1c0","Type":"ContainerStarted","Data":"3dbc05106fa6eb20490c7eb78f3f532f8ebf081c20b82a969cd331ebc07ed87b"} Jul 05 00:17:55.500183 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:55.500134 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-w4fvb" podStartSLOduration=1.6967938729999998 podStartE2EDuration="4.500117308s" totalImagesPullingTime="2.803323435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:17:55.498809788 +0000 UTC m=+513.435175167" watchObservedRunningTime="2026-07-05 00:17:55.500117308 +0000 UTC m=+513.436482687" Jul 05 00:17:55.831641 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:17:55.831551 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:18:02.311035 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:02.310993 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:18:02.411963 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:02.411932 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:18:03.202455 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:03.202419 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:18:03.304061 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:03.304032 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:18:05.226920 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:05.226878 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ml8r4"] Jul 05 00:18:05.229962 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:05.229946 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ml8r4" Jul 05 00:18:05.232603 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:05.232578 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 05 00:18:05.232730 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:05.232701 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-pz7t6\"" Jul 05 00:18:05.232793 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:05.232773 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 05 00:18:05.237238 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:05.237170 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ml8r4"] Jul 05 00:18:05.268730 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:05.268697 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c7f13da-2f23-492b-a34d-970e1ec64026-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ml8r4\" (UID: \"6c7f13da-2f23-492b-a34d-970e1ec64026\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ml8r4" Jul 05 00:18:05.268870 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:05.268748 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq874\" (UniqueName: \"kubernetes.io/projected/6c7f13da-2f23-492b-a34d-970e1ec64026-kube-api-access-qq874\") pod \"openshift-lws-operator-fd8ccff4c-ml8r4\" (UID: \"6c7f13da-2f23-492b-a34d-970e1ec64026\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ml8r4" Jul 05 00:18:05.369650 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:05.369613 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c7f13da-2f23-492b-a34d-970e1ec64026-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ml8r4\" (UID: \"6c7f13da-2f23-492b-a34d-970e1ec64026\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ml8r4" Jul 05 00:18:05.369800 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:05.369670 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq874\" (UniqueName: \"kubernetes.io/projected/6c7f13da-2f23-492b-a34d-970e1ec64026-kube-api-access-qq874\") pod \"openshift-lws-operator-fd8ccff4c-ml8r4\" (UID: \"6c7f13da-2f23-492b-a34d-970e1ec64026\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ml8r4" Jul 05 00:18:05.370112 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:05.370095 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c7f13da-2f23-492b-a34d-970e1ec64026-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ml8r4\" (UID: \"6c7f13da-2f23-492b-a34d-970e1ec64026\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ml8r4" Jul 05 00:18:05.378056 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:05.378027 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq874\" (UniqueName: \"kubernetes.io/projected/6c7f13da-2f23-492b-a34d-970e1ec64026-kube-api-access-qq874\") pod \"openshift-lws-operator-fd8ccff4c-ml8r4\" (UID: \"6c7f13da-2f23-492b-a34d-970e1ec64026\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ml8r4" Jul 05 00:18:05.540135 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:05.540063 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ml8r4" Jul 05 00:18:05.672380 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:05.672357 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ml8r4"] Jul 05 00:18:05.674965 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:18:05.674939 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c7f13da_2f23_492b_a34d_970e1ec64026.slice/crio-8daab5495f1867032f71e7fc694670af6e9f992dd2570a0e2b39942243b14d30 WatchSource:0}: Error finding container 8daab5495f1867032f71e7fc694670af6e9f992dd2570a0e2b39942243b14d30: Status 404 returned error can't find the container with id 8daab5495f1867032f71e7fc694670af6e9f992dd2570a0e2b39942243b14d30 Jul 05 00:18:06.524125 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:06.524091 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ml8r4" event={"ID":"6c7f13da-2f23-492b-a34d-970e1ec64026","Type":"ContainerStarted","Data":"8daab5495f1867032f71e7fc694670af6e9f992dd2570a0e2b39942243b14d30"} Jul 05 00:18:34.637976 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:34.637933 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ml8r4" event={"ID":"6c7f13da-2f23-492b-a34d-970e1ec64026","Type":"ContainerStarted","Data":"49550f48b77cc893ef6e71e65b31fe0704f64c7bdceffb6dc9f6a708108f2250"} Jul 05 00:18:34.656123 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:34.656079 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ml8r4" podStartSLOduration=1.3148330160000001 podStartE2EDuration="29.656063701s" totalImagesPullingTime="28.341230685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:18:34.654410214 +0000 UTC m=+552.590775606" watchObservedRunningTime="2026-07-05 00:18:34.656063701 +0000 UTC m=+552.592429081" Jul 05 00:18:35.357250 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:35.357216 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:18:35.458217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:35.458183 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:18:37.596138 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:37.596104 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:18:41.815177 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:41.815128 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:18:41.891375 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:18:41.891339 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:01.614498 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:01.614467 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:02.549929 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.549884 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8"] Jul 05 00:19:02.553117 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.553098 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.556103 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.556081 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 05 00:19:02.556222 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.556083 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 05 00:19:02.556222 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.556127 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5n2nm\"" Jul 05 00:19:02.556378 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.556362 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 05 00:19:02.556378 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.556374 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 05 00:19:02.567766 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.567744 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8"] Jul 05 00:19:02.605218 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.605187 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-manifests\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.605347 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.605235 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.605347 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.605275 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr5t5\" (UniqueName: \"kubernetes.io/projected/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-kube-api-access-sr5t5\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.605347 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.605339 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-webhook-cert\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.605460 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.605366 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-tmp\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.706139 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.706107 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-manifests\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.706556 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.706156 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.706556 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.706180 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr5t5\" (UniqueName: \"kubernetes.io/projected/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-kube-api-access-sr5t5\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.706556 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.706210 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-webhook-cert\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.706556 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.706333 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-tmp\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.706556 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.706488 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-manifests\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.708573 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.708556 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-tmp\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.708762 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.708740 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-webhook-cert\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.708852 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.708838 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.715204 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.715182 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr5t5\" (UniqueName: \"kubernetes.io/projected/8a6ad246-4f69-4fbf-9cd2-fed27a05502e-kube-api-access-sr5t5\") pod \"opendatahub-operator-controller-manager-674c4bd948-84wl8\" (UID: \"8a6ad246-4f69-4fbf-9cd2-fed27a05502e\") " pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.863073 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.862991 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:02.921762 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.921730 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:02.997713 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:02.997682 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8"] Jul 05 00:19:03.001037 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:19:03.001013 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a6ad246_4f69_4fbf_9cd2_fed27a05502e.slice/crio-4d9da2e7ddb7cc3f08767ba4601a572882c17ac19d14d8adea2af4aee266007a WatchSource:0}: Error finding container 4d9da2e7ddb7cc3f08767ba4601a572882c17ac19d14d8adea2af4aee266007a: Status 404 returned error can't find the container with id 4d9da2e7ddb7cc3f08767ba4601a572882c17ac19d14d8adea2af4aee266007a Jul 05 00:19:03.002317 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:03.002290 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:03.067596 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:03.067568 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:03.166351 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:03.166322 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:03.735912 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:03.735864 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" event={"ID":"8a6ad246-4f69-4fbf-9cd2-fed27a05502e","Type":"ContainerStarted","Data":"4d9da2e7ddb7cc3f08767ba4601a572882c17ac19d14d8adea2af4aee266007a"} Jul 05 00:19:06.750359 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:06.750325 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" event={"ID":"8a6ad246-4f69-4fbf-9cd2-fed27a05502e","Type":"ContainerStarted","Data":"24eec0c1c214bb3fed4175f3799c8e69a0c9dc586d7e45c6e9f36e46c3ebf855"} Jul 05 00:19:07.756111 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:07.756079 2558 generic.go:356] "Generic (PLEG): container finished" podID="8a6ad246-4f69-4fbf-9cd2-fed27a05502e" containerID="24eec0c1c214bb3fed4175f3799c8e69a0c9dc586d7e45c6e9f36e46c3ebf855" exitCode=0 Jul 05 00:19:07.756477 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:07.756135 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" event={"ID":"8a6ad246-4f69-4fbf-9cd2-fed27a05502e","Type":"ContainerDied","Data":"24eec0c1c214bb3fed4175f3799c8e69a0c9dc586d7e45c6e9f36e46c3ebf855"} Jul 05 00:19:08.761980 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:08.761941 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" event={"ID":"8a6ad246-4f69-4fbf-9cd2-fed27a05502e","Type":"ContainerStarted","Data":"63bf6c5ec9b80528fad8ef65c0a6b1d2d093a82ad0a8b343e043c9f31ec36f9b"} Jul 05 00:19:08.762366 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:08.762060 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:08.785175 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:08.785129 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" podStartSLOduration=2.432409797 podStartE2EDuration="6.785114041s" totalImagesPullingTime="3.254906874s" totalInitContainerRuntime="1.09779737s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:19:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:19:08.783811727 +0000 UTC m=+586.720177124" watchObservedRunningTime="2026-07-05 00:19:08.785114041 +0000 UTC m=+586.721479423" Jul 05 00:19:10.314842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:10.314805 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:10.369128 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:10.369101 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:18.725208 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:18.725167 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:18.775300 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:18.775270 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:19.154227 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:19.154188 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:19.206499 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:19.206473 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:19.768553 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:19.768527 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-674c4bd948-84wl8" Jul 05 00:19:20.777404 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:20.777373 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:20.883171 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:20.883143 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:21.231354 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:21.231324 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:21.325795 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:21.325761 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:21.941687 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:21.941656 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dzcz4"] Jul 05 00:19:21.943806 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:21.943792 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dzcz4" Jul 05 00:19:21.946299 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:21.946279 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 05 00:19:21.946487 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:21.946472 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-grfxx\"" Jul 05 00:19:21.953597 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:21.953578 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dzcz4"] Jul 05 00:19:22.051183 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.051153 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:22.056346 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.056323 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28b5b72c-0953-4c98-a9a1-d45eab3be5d3-cert\") pod \"odh-model-controller-7998c67dc4-dzcz4\" (UID: \"28b5b72c-0953-4c98-a9a1-d45eab3be5d3\") " pod="opendatahub/odh-model-controller-7998c67dc4-dzcz4" Jul 05 00:19:22.056479 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.056387 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nr2h\" (UniqueName: \"kubernetes.io/projected/28b5b72c-0953-4c98-a9a1-d45eab3be5d3-kube-api-access-6nr2h\") pod \"odh-model-controller-7998c67dc4-dzcz4\" (UID: \"28b5b72c-0953-4c98-a9a1-d45eab3be5d3\") " pod="opendatahub/odh-model-controller-7998c67dc4-dzcz4" Jul 05 00:19:22.153004 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.152979 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:22.157086 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.157068 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nr2h\" (UniqueName: \"kubernetes.io/projected/28b5b72c-0953-4c98-a9a1-d45eab3be5d3-kube-api-access-6nr2h\") pod \"odh-model-controller-7998c67dc4-dzcz4\" (UID: \"28b5b72c-0953-4c98-a9a1-d45eab3be5d3\") " pod="opendatahub/odh-model-controller-7998c67dc4-dzcz4" Jul 05 00:19:22.157179 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.157125 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28b5b72c-0953-4c98-a9a1-d45eab3be5d3-cert\") pod \"odh-model-controller-7998c67dc4-dzcz4\" (UID: \"28b5b72c-0953-4c98-a9a1-d45eab3be5d3\") " pod="opendatahub/odh-model-controller-7998c67dc4-dzcz4" Jul 05 00:19:22.157227 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:19:22.157216 2558 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 05 00:19:22.157280 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:19:22.157271 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b5b72c-0953-4c98-a9a1-d45eab3be5d3-cert podName:28b5b72c-0953-4c98-a9a1-d45eab3be5d3 nodeName:}" failed. No retries permitted until 2026-07-05 00:19:22.657257025 +0000 UTC m=+600.593622382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/28b5b72c-0953-4c98-a9a1-d45eab3be5d3-cert") pod "odh-model-controller-7998c67dc4-dzcz4" (UID: "28b5b72c-0953-4c98-a9a1-d45eab3be5d3") : secret "odh-model-controller-webhook-cert" not found Jul 05 00:19:22.165804 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.165784 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nr2h\" (UniqueName: \"kubernetes.io/projected/28b5b72c-0953-4c98-a9a1-d45eab3be5d3-kube-api-access-6nr2h\") pod \"odh-model-controller-7998c67dc4-dzcz4\" (UID: \"28b5b72c-0953-4c98-a9a1-d45eab3be5d3\") " pod="opendatahub/odh-model-controller-7998c67dc4-dzcz4" Jul 05 00:19:22.254468 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.254388 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:22.356195 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.356156 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:22.461363 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.461337 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bm469"] Jul 05 00:19:22.463589 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.463573 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bm469" Jul 05 00:19:22.466023 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.466001 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 05 00:19:22.473289 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.473268 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bm469"] Jul 05 00:19:22.499292 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.499266 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:22.559777 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.559708 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10e57536-ef8d-4e8c-a817-907de7fe3366-tls-certs\") pod \"model-serving-api-54d89bd5fc-bm469\" (UID: \"10e57536-ef8d-4e8c-a817-907de7fe3366\") " pod="opendatahub/model-serving-api-54d89bd5fc-bm469" Jul 05 00:19:22.559777 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.559757 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgkpc\" (UniqueName: \"kubernetes.io/projected/10e57536-ef8d-4e8c-a817-907de7fe3366-kube-api-access-tgkpc\") pod \"model-serving-api-54d89bd5fc-bm469\" (UID: \"10e57536-ef8d-4e8c-a817-907de7fe3366\") " pod="opendatahub/model-serving-api-54d89bd5fc-bm469" Jul 05 00:19:22.660162 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.660123 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgkpc\" (UniqueName: \"kubernetes.io/projected/10e57536-ef8d-4e8c-a817-907de7fe3366-kube-api-access-tgkpc\") pod \"model-serving-api-54d89bd5fc-bm469\" (UID: \"10e57536-ef8d-4e8c-a817-907de7fe3366\") " pod="opendatahub/model-serving-api-54d89bd5fc-bm469" Jul 05 00:19:22.660320 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.660182 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28b5b72c-0953-4c98-a9a1-d45eab3be5d3-cert\") pod \"odh-model-controller-7998c67dc4-dzcz4\" (UID: \"28b5b72c-0953-4c98-a9a1-d45eab3be5d3\") " pod="opendatahub/odh-model-controller-7998c67dc4-dzcz4" Jul 05 00:19:22.660320 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.660204 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10e57536-ef8d-4e8c-a817-907de7fe3366-tls-certs\") pod \"model-serving-api-54d89bd5fc-bm469\" (UID: \"10e57536-ef8d-4e8c-a817-907de7fe3366\") " pod="opendatahub/model-serving-api-54d89bd5fc-bm469" Jul 05 00:19:22.660320 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:19:22.660290 2558 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 05 00:19:22.660477 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:19:22.660334 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10e57536-ef8d-4e8c-a817-907de7fe3366-tls-certs podName:10e57536-ef8d-4e8c-a817-907de7fe3366 nodeName:}" failed. No retries permitted until 2026-07-05 00:19:23.160321731 +0000 UTC m=+601.096687088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/10e57536-ef8d-4e8c-a817-907de7fe3366-tls-certs") pod "model-serving-api-54d89bd5fc-bm469" (UID: "10e57536-ef8d-4e8c-a817-907de7fe3366") : secret "model-serving-api-tls" not found Jul 05 00:19:22.662594 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.662562 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28b5b72c-0953-4c98-a9a1-d45eab3be5d3-cert\") pod \"odh-model-controller-7998c67dc4-dzcz4\" (UID: \"28b5b72c-0953-4c98-a9a1-d45eab3be5d3\") " pod="opendatahub/odh-model-controller-7998c67dc4-dzcz4" Jul 05 00:19:22.670130 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.670107 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgkpc\" (UniqueName: \"kubernetes.io/projected/10e57536-ef8d-4e8c-a817-907de7fe3366-kube-api-access-tgkpc\") pod \"model-serving-api-54d89bd5fc-bm469\" (UID: \"10e57536-ef8d-4e8c-a817-907de7fe3366\") " pod="opendatahub/model-serving-api-54d89bd5fc-bm469" Jul 05 00:19:22.703933 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.703886 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:22.768574 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.768545 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:22.834799 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.834731 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:22.856229 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.856210 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-grfxx\"" Jul 05 00:19:22.863891 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.863874 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dzcz4" Jul 05 00:19:22.989636 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:22.989614 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dzcz4"] Jul 05 00:19:22.992100 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:19:22.992071 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28b5b72c_0953_4c98_a9a1_d45eab3be5d3.slice/crio-7819e35281264b1d768a70cbea362f1c8e026408c7a8668975b148a6fe8b40a0 WatchSource:0}: Error finding container 7819e35281264b1d768a70cbea362f1c8e026408c7a8668975b148a6fe8b40a0: Status 404 returned error can't find the container with id 7819e35281264b1d768a70cbea362f1c8e026408c7a8668975b148a6fe8b40a0 Jul 05 00:19:23.165199 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:23.165169 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10e57536-ef8d-4e8c-a817-907de7fe3366-tls-certs\") pod \"model-serving-api-54d89bd5fc-bm469\" (UID: \"10e57536-ef8d-4e8c-a817-907de7fe3366\") " pod="opendatahub/model-serving-api-54d89bd5fc-bm469" Jul 05 00:19:23.167571 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:23.167544 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10e57536-ef8d-4e8c-a817-907de7fe3366-tls-certs\") pod \"model-serving-api-54d89bd5fc-bm469\" (UID: \"10e57536-ef8d-4e8c-a817-907de7fe3366\") " pod="opendatahub/model-serving-api-54d89bd5fc-bm469" Jul 05 00:19:23.375015 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:23.374985 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bm469" Jul 05 00:19:23.499590 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:23.499512 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bm469"] Jul 05 00:19:23.502080 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:19:23.502054 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10e57536_ef8d_4e8c_a817_907de7fe3366.slice/crio-b4579b488ea01157d50cff696d5300b68c1dd4567d474d5b8b09bb2e0dcd7222 WatchSource:0}: Error finding container b4579b488ea01157d50cff696d5300b68c1dd4567d474d5b8b09bb2e0dcd7222: Status 404 returned error can't find the container with id b4579b488ea01157d50cff696d5300b68c1dd4567d474d5b8b09bb2e0dcd7222 Jul 05 00:19:23.830566 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:23.830477 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bm469" event={"ID":"10e57536-ef8d-4e8c-a817-907de7fe3366","Type":"ContainerStarted","Data":"b4579b488ea01157d50cff696d5300b68c1dd4567d474d5b8b09bb2e0dcd7222"} Jul 05 00:19:23.832916 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:23.832865 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dzcz4" event={"ID":"28b5b72c-0953-4c98-a9a1-d45eab3be5d3","Type":"ContainerStarted","Data":"7819e35281264b1d768a70cbea362f1c8e026408c7a8668975b148a6fe8b40a0"} Jul 05 00:19:26.849866 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:26.849828 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bm469" event={"ID":"10e57536-ef8d-4e8c-a817-907de7fe3366","Type":"ContainerStarted","Data":"95ecb6d999f06982a938509b8eab234669adf050e50b307601700ea9708a986d"} Jul 05 00:19:26.850400 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:26.849967 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-bm469" Jul 05 00:19:26.851948 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:26.851897 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dzcz4" event={"ID":"28b5b72c-0953-4c98-a9a1-d45eab3be5d3","Type":"ContainerStarted","Data":"d55bfe1b25df4fc0af14e07b9006277193495b068284d04a562bfad44258ef97"} Jul 05 00:19:26.852078 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:26.851974 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-dzcz4" Jul 05 00:19:26.867034 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:26.866981 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-bm469" podStartSLOduration=2.157867637 podStartE2EDuration="4.866968373s" totalImagesPullingTime="2.709100736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:19:26.865314727 +0000 UTC m=+604.801680105" watchObservedRunningTime="2026-07-05 00:19:26.866968373 +0000 UTC m=+604.803333752" Jul 05 00:19:26.879848 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:26.879807 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-dzcz4" podStartSLOduration=2.65633997 podStartE2EDuration="5.8797957s" totalImagesPullingTime="3.22345573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:19:26.878420693 +0000 UTC m=+604.814786072" watchObservedRunningTime="2026-07-05 00:19:26.8797957 +0000 UTC m=+604.816161079" Jul 05 00:19:30.686209 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.686173 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:30.810666 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.810636 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-79bvr"] Jul 05 00:19:30.812764 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.812749 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-79bvr" Jul 05 00:19:30.815367 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.815347 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 05 00:19:30.815466 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.815435 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jkv5f\"" Jul 05 00:19:30.819112 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.819080 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:30.819687 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.819667 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6"] Jul 05 00:19:30.821702 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.821688 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6" Jul 05 00:19:30.824389 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.824369 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 05 00:19:30.824389 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.824382 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-jpmsc\"" Jul 05 00:19:30.824708 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.824692 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-79bvr"] Jul 05 00:19:30.831937 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.831890 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6"] Jul 05 00:19:30.938308 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.938239 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e9b555f-2665-44e7-95b2-9587f4863760-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vgkg6\" (UID: \"3e9b555f-2665-44e7-95b2-9587f4863760\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6" Jul 05 00:19:30.938308 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.938289 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl76x\" (UniqueName: \"kubernetes.io/projected/a25ed367-e43c-4191-86d6-557cb7f14e4a-kube-api-access-rl76x\") pod \"kserve-controller-manager-d6f4b65b8-79bvr\" (UID: \"a25ed367-e43c-4191-86d6-557cb7f14e4a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-79bvr" Jul 05 00:19:30.938469 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.938348 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a25ed367-e43c-4191-86d6-557cb7f14e4a-cert\") pod \"kserve-controller-manager-d6f4b65b8-79bvr\" (UID: \"a25ed367-e43c-4191-86d6-557cb7f14e4a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-79bvr" Jul 05 00:19:30.938469 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:30.938408 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkjx7\" (UniqueName: \"kubernetes.io/projected/3e9b555f-2665-44e7-95b2-9587f4863760-kube-api-access-bkjx7\") pod \"llmisvc-controller-manager-777c6bd7d5-vgkg6\" (UID: \"3e9b555f-2665-44e7-95b2-9587f4863760\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6" Jul 05 00:19:31.038952 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.038920 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e9b555f-2665-44e7-95b2-9587f4863760-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vgkg6\" (UID: \"3e9b555f-2665-44e7-95b2-9587f4863760\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6" Jul 05 00:19:31.039078 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.038974 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl76x\" (UniqueName: \"kubernetes.io/projected/a25ed367-e43c-4191-86d6-557cb7f14e4a-kube-api-access-rl76x\") pod \"kserve-controller-manager-d6f4b65b8-79bvr\" (UID: \"a25ed367-e43c-4191-86d6-557cb7f14e4a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-79bvr" Jul 05 00:19:31.039078 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.039000 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a25ed367-e43c-4191-86d6-557cb7f14e4a-cert\") pod \"kserve-controller-manager-d6f4b65b8-79bvr\" (UID: \"a25ed367-e43c-4191-86d6-557cb7f14e4a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-79bvr" Jul 05 00:19:31.039078 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.039047 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkjx7\" (UniqueName: \"kubernetes.io/projected/3e9b555f-2665-44e7-95b2-9587f4863760-kube-api-access-bkjx7\") pod \"llmisvc-controller-manager-777c6bd7d5-vgkg6\" (UID: \"3e9b555f-2665-44e7-95b2-9587f4863760\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6" Jul 05 00:19:31.039078 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:19:31.039057 2558 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 05 00:19:31.039279 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:19:31.039108 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e9b555f-2665-44e7-95b2-9587f4863760-cert podName:3e9b555f-2665-44e7-95b2-9587f4863760 nodeName:}" failed. No retries permitted until 2026-07-05 00:19:31.539093771 +0000 UTC m=+609.475459128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3e9b555f-2665-44e7-95b2-9587f4863760-cert") pod "llmisvc-controller-manager-777c6bd7d5-vgkg6" (UID: "3e9b555f-2665-44e7-95b2-9587f4863760") : secret "llmisvc-webhook-server-cert" not found Jul 05 00:19:31.041680 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.041652 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a25ed367-e43c-4191-86d6-557cb7f14e4a-cert\") pod \"kserve-controller-manager-d6f4b65b8-79bvr\" (UID: \"a25ed367-e43c-4191-86d6-557cb7f14e4a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-79bvr" Jul 05 00:19:31.047842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.047817 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkjx7\" (UniqueName: \"kubernetes.io/projected/3e9b555f-2665-44e7-95b2-9587f4863760-kube-api-access-bkjx7\") pod \"llmisvc-controller-manager-777c6bd7d5-vgkg6\" (UID: \"3e9b555f-2665-44e7-95b2-9587f4863760\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6" Jul 05 00:19:31.048042 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.048023 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl76x\" (UniqueName: \"kubernetes.io/projected/a25ed367-e43c-4191-86d6-557cb7f14e4a-kube-api-access-rl76x\") pod \"kserve-controller-manager-d6f4b65b8-79bvr\" (UID: \"a25ed367-e43c-4191-86d6-557cb7f14e4a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-79bvr" Jul 05 00:19:31.069059 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.069036 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:31.123134 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.123112 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-79bvr" Jul 05 00:19:31.212154 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.212128 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:31.264763 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.264744 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-79bvr"] Jul 05 00:19:31.266875 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:19:31.266851 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25ed367_e43c_4191_86d6_557cb7f14e4a.slice/crio-218b409bd786e4a79669b7c5c325743d7f294993aa7c180599cf2a0c8404305a WatchSource:0}: Error finding container 218b409bd786e4a79669b7c5c325743d7f294993aa7c180599cf2a0c8404305a: Status 404 returned error can't find the container with id 218b409bd786e4a79669b7c5c325743d7f294993aa7c180599cf2a0c8404305a Jul 05 00:19:31.543736 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.543651 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e9b555f-2665-44e7-95b2-9587f4863760-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vgkg6\" (UID: \"3e9b555f-2665-44e7-95b2-9587f4863760\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6" Jul 05 00:19:31.546142 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.546124 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e9b555f-2665-44e7-95b2-9587f4863760-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vgkg6\" (UID: \"3e9b555f-2665-44e7-95b2-9587f4863760\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6" Jul 05 00:19:31.732213 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.732183 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6" Jul 05 00:19:31.863183 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.863142 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6"] Jul 05 00:19:31.866991 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:19:31.866961 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3e9b555f_2665_44e7_95b2_9587f4863760.slice/crio-092935d966a844da2c226f76edb52dc5949f9002d7a6e91113669a5a4ef5f4d0 WatchSource:0}: Error finding container 092935d966a844da2c226f76edb52dc5949f9002d7a6e91113669a5a4ef5f4d0: Status 404 returned error can't find the container with id 092935d966a844da2c226f76edb52dc5949f9002d7a6e91113669a5a4ef5f4d0 Jul 05 00:19:31.874788 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.874758 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6" event={"ID":"3e9b555f-2665-44e7-95b2-9587f4863760","Type":"ContainerStarted","Data":"092935d966a844da2c226f76edb52dc5949f9002d7a6e91113669a5a4ef5f4d0"} Jul 05 00:19:31.876038 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.876014 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-79bvr" event={"ID":"a25ed367-e43c-4191-86d6-557cb7f14e4a","Type":"ContainerStarted","Data":"218b409bd786e4a79669b7c5c325743d7f294993aa7c180599cf2a0c8404305a"} Jul 05 00:19:31.895848 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.895818 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:31.928039 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.928007 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl"] Jul 05 00:19:31.930769 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.930747 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" Jul 05 00:19:31.933541 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.933512 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 05 00:19:31.934873 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.934849 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 05 00:19:31.935048 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.934990 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-nw2x6\"" Jul 05 00:19:31.935048 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.934990 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 05 00:19:31.935223 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.935068 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 05 00:19:31.942559 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:31.942538 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl"] Jul 05 00:19:32.035042 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:32.034990 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:32.048783 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:32.048759 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k9jb\" (UniqueName: \"kubernetes.io/projected/922beef2-8ec5-40ae-89c0-ce53ff159809-kube-api-access-9k9jb\") pod \"kube-auth-proxy-6bddf4ddb8-zhtcl\" (UID: \"922beef2-8ec5-40ae-89c0-ce53ff159809\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" Jul 05 00:19:32.048988 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:32.048804 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/922beef2-8ec5-40ae-89c0-ce53ff159809-tmp\") pod \"kube-auth-proxy-6bddf4ddb8-zhtcl\" (UID: \"922beef2-8ec5-40ae-89c0-ce53ff159809\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" Jul 05 00:19:32.048988 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:32.048849 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/922beef2-8ec5-40ae-89c0-ce53ff159809-tls-certs\") pod \"kube-auth-proxy-6bddf4ddb8-zhtcl\" (UID: \"922beef2-8ec5-40ae-89c0-ce53ff159809\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" Jul 05 00:19:32.145355 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:32.145324 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:32.149340 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:32.149305 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/922beef2-8ec5-40ae-89c0-ce53ff159809-tmp\") pod \"kube-auth-proxy-6bddf4ddb8-zhtcl\" (UID: \"922beef2-8ec5-40ae-89c0-ce53ff159809\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" Jul 05 00:19:32.149454 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:32.149360 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/922beef2-8ec5-40ae-89c0-ce53ff159809-tls-certs\") pod \"kube-auth-proxy-6bddf4ddb8-zhtcl\" (UID: \"922beef2-8ec5-40ae-89c0-ce53ff159809\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" Jul 05 00:19:32.149454 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:32.149436 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k9jb\" (UniqueName: \"kubernetes.io/projected/922beef2-8ec5-40ae-89c0-ce53ff159809-kube-api-access-9k9jb\") pod \"kube-auth-proxy-6bddf4ddb8-zhtcl\" (UID: \"922beef2-8ec5-40ae-89c0-ce53ff159809\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" Jul 05 00:19:32.149566 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:19:32.149518 2558 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 05 00:19:32.149609 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:19:32.149584 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/922beef2-8ec5-40ae-89c0-ce53ff159809-tls-certs podName:922beef2-8ec5-40ae-89c0-ce53ff159809 nodeName:}" failed. No retries permitted until 2026-07-05 00:19:32.649564203 +0000 UTC m=+610.585929575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/922beef2-8ec5-40ae-89c0-ce53ff159809-tls-certs") pod "kube-auth-proxy-6bddf4ddb8-zhtcl" (UID: "922beef2-8ec5-40ae-89c0-ce53ff159809") : secret "kube-auth-proxy-tls" not found Jul 05 00:19:32.152107 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:32.152084 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/922beef2-8ec5-40ae-89c0-ce53ff159809-tmp\") pod \"kube-auth-proxy-6bddf4ddb8-zhtcl\" (UID: \"922beef2-8ec5-40ae-89c0-ce53ff159809\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" Jul 05 00:19:32.160757 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:32.160713 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k9jb\" (UniqueName: \"kubernetes.io/projected/922beef2-8ec5-40ae-89c0-ce53ff159809-kube-api-access-9k9jb\") pod \"kube-auth-proxy-6bddf4ddb8-zhtcl\" (UID: \"922beef2-8ec5-40ae-89c0-ce53ff159809\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" Jul 05 00:19:32.510222 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:32.510158 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:32.653777 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:32.653748 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/922beef2-8ec5-40ae-89c0-ce53ff159809-tls-certs\") pod \"kube-auth-proxy-6bddf4ddb8-zhtcl\" (UID: \"922beef2-8ec5-40ae-89c0-ce53ff159809\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" Jul 05 00:19:32.656967 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:32.656844 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/922beef2-8ec5-40ae-89c0-ce53ff159809-tls-certs\") pod \"kube-auth-proxy-6bddf4ddb8-zhtcl\" (UID: \"922beef2-8ec5-40ae-89c0-ce53ff159809\") " pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" Jul 05 00:19:32.847664 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:32.847588 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" Jul 05 00:19:33.299538 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.299500 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl"] Jul 05 00:19:33.444078 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.444001 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc"] Jul 05 00:19:33.447955 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.447929 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:33.448620 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.448602 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.451247 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.451214 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 05 00:19:33.451247 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.451231 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 05 00:19:33.451247 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.451219 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 05 00:19:33.451875 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.451612 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 05 00:19:33.451875 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.451715 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-r6zt2\"" Jul 05 00:19:33.457751 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.457697 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc"] Jul 05 00:19:33.516856 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.516819 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:33.563418 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.563387 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.563581 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.563477 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.563581 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.563525 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.563581 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.563564 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqlnc\" (UniqueName: \"kubernetes.io/projected/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-kube-api-access-vqlnc\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.563737 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.563601 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.563737 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.563633 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.563737 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.563656 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.664110 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.664080 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.664302 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.664124 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.664302 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.664165 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqlnc\" (UniqueName: \"kubernetes.io/projected/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-kube-api-access-vqlnc\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.664302 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.664206 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.664302 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.664236 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.664302 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.664260 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.664302 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.664297 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.665099 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.665024 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.667283 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.667217 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.667283 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.667223 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.667458 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.667298 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.667458 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.667398 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.673302 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.673280 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.673677 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.673657 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqlnc\" (UniqueName: \"kubernetes.io/projected/50249fa7-6685-4f7d-9ba2-feed1eedcbf0-kube-api-access-vqlnc\") pod \"istiod-openshift-gateway-94bb8fbfd-fsbtc\" (UID: \"50249fa7-6685-4f7d-9ba2-feed1eedcbf0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.761745 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.761654 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:33.868971 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:19:33.868936 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922beef2_8ec5_40ae_89c0_ce53ff159809.slice/crio-ad26c6d250c980a4341ce349a02a5499ce6b103bd61bee9c9e6fd5945fb998c9 WatchSource:0}: Error finding container ad26c6d250c980a4341ce349a02a5499ce6b103bd61bee9c9e6fd5945fb998c9: Status 404 returned error can't find the container with id ad26c6d250c980a4341ce349a02a5499ce6b103bd61bee9c9e6fd5945fb998c9 Jul 05 00:19:33.887235 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:33.887200 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" event={"ID":"922beef2-8ec5-40ae-89c0-ce53ff159809","Type":"ContainerStarted","Data":"ad26c6d250c980a4341ce349a02a5499ce6b103bd61bee9c9e6fd5945fb998c9"} Jul 05 00:19:34.003700 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:34.003621 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc"] Jul 05 00:19:34.005798 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:19:34.005722 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50249fa7_6685_4f7d_9ba2_feed1eedcbf0.slice/crio-30b49020356944724d409d98105745eee513329a6f7afcf029a9587e79c21cba WatchSource:0}: Error finding container 30b49020356944724d409d98105745eee513329a6f7afcf029a9587e79c21cba: Status 404 returned error can't find the container with id 30b49020356944724d409d98105745eee513329a6f7afcf029a9587e79c21cba Jul 05 00:19:34.500156 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:34.500118 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:34.893642 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:34.893609 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" event={"ID":"50249fa7-6685-4f7d-9ba2-feed1eedcbf0","Type":"ContainerStarted","Data":"30b49020356944724d409d98105745eee513329a6f7afcf029a9587e79c21cba"} Jul 05 00:19:34.895508 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:34.895484 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6" event={"ID":"3e9b555f-2665-44e7-95b2-9587f4863760","Type":"ContainerStarted","Data":"de38fb15c4a8be7ab856a60a42da21de09ef22e326b671158abba5c61ad5521b"} Jul 05 00:19:34.895620 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:34.895537 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6" Jul 05 00:19:34.897249 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:34.897227 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-79bvr" event={"ID":"a25ed367-e43c-4191-86d6-557cb7f14e4a","Type":"ContainerStarted","Data":"3fef58a01dd0036346997780c28d0ba20b866f356825fdac9fe07a1695b6508b"} Jul 05 00:19:34.897383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:34.897363 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-79bvr" Jul 05 00:19:34.912406 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:34.912361 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6" podStartSLOduration=1.991242846 podStartE2EDuration="4.912343149s" totalImagesPullingTime="2.921100303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:19:34.911191785 +0000 UTC m=+612.847557196" watchObservedRunningTime="2026-07-05 00:19:34.912343149 +0000 UTC m=+612.848708530" Jul 05 00:19:34.927531 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:34.927496 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-79bvr" podStartSLOduration=2.275543363 podStartE2EDuration="4.927486895s" totalImagesPullingTime="2.651943532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:19:34.925788354 +0000 UTC m=+612.862153760" watchObservedRunningTime="2026-07-05 00:19:34.927486895 +0000 UTC m=+612.863852274" Jul 05 00:19:37.484375 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:37.484341 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:37.858502 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:37.858432 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-dzcz4" Jul 05 00:19:37.860492 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:37.860473 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-bm469" Jul 05 00:19:37.990329 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:37.990292 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 05 00:19:37.990439 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:37.990369 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 05 00:19:38.924337 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:38.924280 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" event={"ID":"50249fa7-6685-4f7d-9ba2-feed1eedcbf0","Type":"ContainerStarted","Data":"532ceb99ab4e64531fa0e8d9b60c0d8c6d9063ffe8a7862a397152a82da63892"} Jul 05 00:19:38.924337 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:38.924341 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:19:38.925870 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:38.925840 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" event={"ID":"922beef2-8ec5-40ae-89c0-ce53ff159809","Type":"ContainerStarted","Data":"9d92ededf721f6f6013899c426766180999f27adff42999247c2b2f72580be3e"} Jul 05 00:19:38.945191 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:38.945150 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" podStartSLOduration=1.963137427 podStartE2EDuration="5.945136611s" totalImagesPullingTime="3.981999184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:19:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:19:38.944150203 +0000 UTC m=+616.880515586" watchObservedRunningTime="2026-07-05 00:19:38.945136611 +0000 UTC m=+616.881501991" Jul 05 00:19:38.958522 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:38.958488 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bddf4ddb8-zhtcl" podStartSLOduration=3.848901023 podStartE2EDuration="7.958475718s" totalImagesPullingTime="4.109574695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:19:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:19:38.958261281 +0000 UTC m=+616.894626679" watchObservedRunningTime="2026-07-05 00:19:38.958475718 +0000 UTC m=+616.894841094" Jul 05 00:19:39.038419 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:39.038380 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:39.101737 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:39.101699 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:39.612376 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:39.612326 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:39.735021 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:39.734985 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:39.923288 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:39.923257 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:19:39.931232 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:19:39.931213 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fsbtc" Jul 05 00:20:05.910651 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:05.910570 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vgkg6" Jul 05 00:20:05.913527 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:05.913510 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-79bvr" Jul 05 00:20:22.691524 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:22.691489 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:22.789405 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:22.789371 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:22.862448 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:22.862420 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:36.477712 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:36.477681 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:36.577732 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:36.577698 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:36.629985 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:36.629954 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:37.203427 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:37.203395 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:37.251970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:37.251943 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:47.710104 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:47.710071 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:47.830234 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:47.830196 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:47.880604 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:47.880577 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:51.656199 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:51.656164 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:52.261313 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:52.261280 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:57.031585 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:57.031554 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:57.150054 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:57.150026 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:59.012547 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:59.012512 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cv5tv"] Jul 05 00:20:59.014849 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:59.014833 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cv5tv" Jul 05 00:20:59.017414 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:59.017395 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 05 00:20:59.017515 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:59.017448 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 05 00:20:59.018725 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:59.018705 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-k5hz4\"" Jul 05 00:20:59.018822 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:59.018805 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 05 00:20:59.026186 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:59.026163 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cv5tv"] Jul 05 00:20:59.075173 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:59.075149 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn65s\" (UniqueName: \"kubernetes.io/projected/93a20e30-fa1f-4676-bbfe-ec0005381944-kube-api-access-kn65s\") pod \"dns-operator-controller-manager-864dd78c4c-cv5tv\" (UID: \"93a20e30-fa1f-4676-bbfe-ec0005381944\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cv5tv" Jul 05 00:20:59.103026 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:59.103000 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:20:59.175850 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:59.175821 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn65s\" (UniqueName: \"kubernetes.io/projected/93a20e30-fa1f-4676-bbfe-ec0005381944-kube-api-access-kn65s\") pod \"dns-operator-controller-manager-864dd78c4c-cv5tv\" (UID: \"93a20e30-fa1f-4676-bbfe-ec0005381944\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cv5tv" Jul 05 00:20:59.184841 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:59.184819 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn65s\" (UniqueName: \"kubernetes.io/projected/93a20e30-fa1f-4676-bbfe-ec0005381944-kube-api-access-kn65s\") pod \"dns-operator-controller-manager-864dd78c4c-cv5tv\" (UID: \"93a20e30-fa1f-4676-bbfe-ec0005381944\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cv5tv" Jul 05 00:20:59.325435 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:59.325351 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cv5tv" Jul 05 00:20:59.458453 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:59.458431 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cv5tv"] Jul 05 00:20:59.461099 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:20:59.461075 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93a20e30_fa1f_4676_bbfe_ec0005381944.slice/crio-6aee3646b2718f394662557969aba0133c34d05922a200deb93332711de10baf WatchSource:0}: Error finding container 6aee3646b2718f394662557969aba0133c34d05922a200deb93332711de10baf: Status 404 returned error can't find the container with id 6aee3646b2718f394662557969aba0133c34d05922a200deb93332711de10baf Jul 05 00:20:59.463133 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:20:59.463117 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 00:21:00.063181 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:00.063143 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:00.265026 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:00.264988 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cv5tv" event={"ID":"93a20e30-fa1f-4676-bbfe-ec0005381944","Type":"ContainerStarted","Data":"6aee3646b2718f394662557969aba0133c34d05922a200deb93332711de10baf"} Jul 05 00:21:00.891435 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:00.891398 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:02.276211 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:02.276173 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cv5tv" event={"ID":"93a20e30-fa1f-4676-bbfe-ec0005381944","Type":"ContainerStarted","Data":"02abd9ad4d5a7bdbd4ddef71b5187b5b206e5c130397529bf289e3d46a58cbf4"} Jul 05 00:21:02.276686 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:02.276294 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cv5tv" Jul 05 00:21:02.293226 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:02.293177 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cv5tv" podStartSLOduration=2.287986706 podStartE2EDuration="4.293162098s" totalImagesPullingTime="2.005175392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:21:02.292323624 +0000 UTC m=+700.228689003" watchObservedRunningTime="2026-07-05 00:21:02.293162098 +0000 UTC m=+700.229527476" Jul 05 00:21:03.067976 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:03.067938 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:03.533940 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:03.533894 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:06.776321 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:06.776287 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:08.672511 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:08.672480 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:08.773660 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:08.773630 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:09.072080 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:09.072003 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:09.170607 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:09.170573 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:12.465525 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:12.465487 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:12.515045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:12.515013 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:13.285332 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:13.285300 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cv5tv" Jul 05 00:21:16.269614 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:16.269584 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:16.372951 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:16.372891 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:16.667462 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:16.667435 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:17.023629 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:17.023517 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:19.786552 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:19.786523 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:19.887964 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:19.887931 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:26.874075 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:26.874037 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk"] Jul 05 00:21:26.876480 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:26.876464 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk" Jul 05 00:21:26.879393 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:26.879370 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-r8hkz\"" Jul 05 00:21:26.887640 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:26.887616 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk"] Jul 05 00:21:27.011458 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.011424 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm5hh\" (UniqueName: \"kubernetes.io/projected/c6bed555-54fb-49cb-aa0d-12d358360e3d-kube-api-access-tm5hh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rt2sk\" (UID: \"c6bed555-54fb-49cb-aa0d-12d358360e3d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk" Jul 05 00:21:27.011649 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.011569 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6bed555-54fb-49cb-aa0d-12d358360e3d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rt2sk\" (UID: \"c6bed555-54fb-49cb-aa0d-12d358360e3d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk" Jul 05 00:21:27.112414 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.112374 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6bed555-54fb-49cb-aa0d-12d358360e3d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rt2sk\" (UID: \"c6bed555-54fb-49cb-aa0d-12d358360e3d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk" Jul 05 00:21:27.112414 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.112419 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm5hh\" (UniqueName: \"kubernetes.io/projected/c6bed555-54fb-49cb-aa0d-12d358360e3d-kube-api-access-tm5hh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rt2sk\" (UID: \"c6bed555-54fb-49cb-aa0d-12d358360e3d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk" Jul 05 00:21:27.112771 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.112739 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6bed555-54fb-49cb-aa0d-12d358360e3d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rt2sk\" (UID: \"c6bed555-54fb-49cb-aa0d-12d358360e3d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk" Jul 05 00:21:27.124500 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.124436 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm5hh\" (UniqueName: \"kubernetes.io/projected/c6bed555-54fb-49cb-aa0d-12d358360e3d-kube-api-access-tm5hh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rt2sk\" (UID: \"c6bed555-54fb-49cb-aa0d-12d358360e3d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk" Jul 05 00:21:27.186924 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.186858 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk" Jul 05 00:21:27.321724 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.321689 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk"] Jul 05 00:21:27.325636 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:21:27.325606 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6bed555_54fb_49cb_aa0d_12d358360e3d.slice/crio-89ee3178c310e9dee6a089788ec876b9ba8c9f4cb3016b5061c72fe09b7a2eea WatchSource:0}: Error finding container 89ee3178c310e9dee6a089788ec876b9ba8c9f4cb3016b5061c72fe09b7a2eea: Status 404 returned error can't find the container with id 89ee3178c310e9dee6a089788ec876b9ba8c9f4cb3016b5061c72fe09b7a2eea Jul 05 00:21:27.385817 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.385743 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk" event={"ID":"c6bed555-54fb-49cb-aa0d-12d358360e3d","Type":"ContainerStarted","Data":"89ee3178c310e9dee6a089788ec876b9ba8c9f4cb3016b5061c72fe09b7a2eea"} Jul 05 00:21:27.534825 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.534794 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk"] Jul 05 00:21:27.537438 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.537416 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk"] Jul 05 00:21:27.558803 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.558757 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5"] Jul 05 00:21:27.562378 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.562356 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" Jul 05 00:21:27.577754 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.577726 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5"] Jul 05 00:21:27.666105 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.666075 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:27.718429 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.718403 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmmc9\" (UniqueName: \"kubernetes.io/projected/6144ffd7-4e3f-4416-9b9a-08b81d6030d2-kube-api-access-lmmc9\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bgcj5\" (UID: \"6144ffd7-4e3f-4416-9b9a-08b81d6030d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" Jul 05 00:21:27.718598 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.718444 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6144ffd7-4e3f-4416-9b9a-08b81d6030d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bgcj5\" (UID: \"6144ffd7-4e3f-4416-9b9a-08b81d6030d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" Jul 05 00:21:27.766767 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.766740 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:27.819793 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.819758 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmmc9\" (UniqueName: \"kubernetes.io/projected/6144ffd7-4e3f-4416-9b9a-08b81d6030d2-kube-api-access-lmmc9\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bgcj5\" (UID: \"6144ffd7-4e3f-4416-9b9a-08b81d6030d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" Jul 05 00:21:27.819967 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.819813 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6144ffd7-4e3f-4416-9b9a-08b81d6030d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bgcj5\" (UID: \"6144ffd7-4e3f-4416-9b9a-08b81d6030d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" Jul 05 00:21:27.820259 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.820242 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6144ffd7-4e3f-4416-9b9a-08b81d6030d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bgcj5\" (UID: \"6144ffd7-4e3f-4416-9b9a-08b81d6030d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" Jul 05 00:21:27.831566 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.831539 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmmc9\" (UniqueName: \"kubernetes.io/projected/6144ffd7-4e3f-4416-9b9a-08b81d6030d2-kube-api-access-lmmc9\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bgcj5\" (UID: \"6144ffd7-4e3f-4416-9b9a-08b81d6030d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" Jul 05 00:21:27.875874 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:27.875842 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" Jul 05 00:21:28.025652 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:28.025619 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5"] Jul 05 00:21:28.043951 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:28.043890 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:28.117475 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:21:28.117442 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6144ffd7_4e3f_4416_9b9a_08b81d6030d2.slice/crio-ccf95bb56f3cbbeaa8caeb45a883d1afa814e80dccefe4feb96a1eee636d1e94 WatchSource:0}: Error finding container ccf95bb56f3cbbeaa8caeb45a883d1afa814e80dccefe4feb96a1eee636d1e94: Status 404 returned error can't find the container with id ccf95bb56f3cbbeaa8caeb45a883d1afa814e80dccefe4feb96a1eee636d1e94 Jul 05 00:21:28.137678 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:28.137437 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:28.393154 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:28.393112 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" event={"ID":"6144ffd7-4e3f-4416-9b9a-08b81d6030d2","Type":"ContainerStarted","Data":"ccf95bb56f3cbbeaa8caeb45a883d1afa814e80dccefe4feb96a1eee636d1e94"} Jul 05 00:21:30.506256 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:30.506224 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:30.618885 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:30.618840 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:30.720029 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:30.719989 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:32.415082 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:32.415037 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk" podUID="c6bed555-54fb-49cb-aa0d-12d358360e3d" containerName="manager" containerID="cri-o://6965792b9de6f03bbc19cce69a07c376c32a2c4e4b469fc1eb9b6ba6eadbbab6" gracePeriod=2 Jul 05 00:21:32.417163 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:32.417138 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" event={"ID":"6144ffd7-4e3f-4416-9b9a-08b81d6030d2","Type":"ContainerStarted","Data":"f0a1c0391896b2e5960dee6c94403f7eefb86868f404f6622d0c4b69a1f27a88"} Jul 05 00:21:32.417280 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:32.417263 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" Jul 05 00:21:32.436757 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:32.436712 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" podStartSLOduration=1.735428544 podStartE2EDuration="5.43669892s" totalImagesPullingTime="3.701270376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:21:32.436197628 +0000 UTC m=+730.372563025" watchObservedRunningTime="2026-07-05 00:21:32.43669892 +0000 UTC m=+730.373064299" Jul 05 00:21:32.661614 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:32.661590 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk" Jul 05 00:21:32.768818 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:32.768740 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tm5hh\" (UniqueName: \"kubernetes.io/projected/c6bed555-54fb-49cb-aa0d-12d358360e3d-kube-api-access-tm5hh\") pod \"c6bed555-54fb-49cb-aa0d-12d358360e3d\" (UID: \"c6bed555-54fb-49cb-aa0d-12d358360e3d\") " Jul 05 00:21:32.768818 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:32.768816 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6bed555-54fb-49cb-aa0d-12d358360e3d-extensions-socket-volume\") pod \"c6bed555-54fb-49cb-aa0d-12d358360e3d\" (UID: \"c6bed555-54fb-49cb-aa0d-12d358360e3d\") " Jul 05 00:21:32.769108 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:32.769080 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6bed555-54fb-49cb-aa0d-12d358360e3d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c6bed555-54fb-49cb-aa0d-12d358360e3d" (UID: "c6bed555-54fb-49cb-aa0d-12d358360e3d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 00:21:32.771079 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:32.771061 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6bed555-54fb-49cb-aa0d-12d358360e3d-kube-api-access-tm5hh" (OuterVolumeSpecName: "kube-api-access-tm5hh") pod "c6bed555-54fb-49cb-aa0d-12d358360e3d" (UID: "c6bed555-54fb-49cb-aa0d-12d358360e3d"). InnerVolumeSpecName "kube-api-access-tm5hh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:21:32.869721 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:32.869693 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tm5hh\" (UniqueName: \"kubernetes.io/projected/c6bed555-54fb-49cb-aa0d-12d358360e3d-kube-api-access-tm5hh\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:21:32.869721 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:32.869716 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6bed555-54fb-49cb-aa0d-12d358360e3d-extensions-socket-volume\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:21:33.423827 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:33.423793 2558 generic.go:356] "Generic (PLEG): container finished" podID="c6bed555-54fb-49cb-aa0d-12d358360e3d" containerID="6965792b9de6f03bbc19cce69a07c376c32a2c4e4b469fc1eb9b6ba6eadbbab6" exitCode=2 Jul 05 00:21:33.424273 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:33.423839 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk" Jul 05 00:21:33.424273 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:33.423896 2558 scope.go:117] "RemoveContainer" containerID="6965792b9de6f03bbc19cce69a07c376c32a2c4e4b469fc1eb9b6ba6eadbbab6" Jul 05 00:21:33.426423 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:21:33.426397 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rt2sk\" is forbidden: User \"system:node:ip-10-0-130-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-0.ec2.internal' and this object" podUID="c6bed555-54fb-49cb-aa0d-12d358360e3d" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk" Jul 05 00:21:33.433141 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:33.432872 2558 scope.go:117] "RemoveContainer" containerID="6965792b9de6f03bbc19cce69a07c376c32a2c4e4b469fc1eb9b6ba6eadbbab6" Jul 05 00:21:33.433253 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:21:33.433229 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6965792b9de6f03bbc19cce69a07c376c32a2c4e4b469fc1eb9b6ba6eadbbab6\": container with ID starting with 6965792b9de6f03bbc19cce69a07c376c32a2c4e4b469fc1eb9b6ba6eadbbab6 not found: ID does not exist" containerID="6965792b9de6f03bbc19cce69a07c376c32a2c4e4b469fc1eb9b6ba6eadbbab6" Jul 05 00:21:33.433316 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:33.433263 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6965792b9de6f03bbc19cce69a07c376c32a2c4e4b469fc1eb9b6ba6eadbbab6"} err="failed to get container status \"6965792b9de6f03bbc19cce69a07c376c32a2c4e4b469fc1eb9b6ba6eadbbab6\": rpc error: code = NotFound desc = could not find container \"6965792b9de6f03bbc19cce69a07c376c32a2c4e4b469fc1eb9b6ba6eadbbab6\": container with ID starting with 6965792b9de6f03bbc19cce69a07c376c32a2c4e4b469fc1eb9b6ba6eadbbab6 not found: ID does not exist" Jul 05 00:21:33.436382 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:21:33.436353 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-rt2sk\" is forbidden: User \"system:node:ip-10-0-130-0.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-0.ec2.internal' and this object" podUID="c6bed555-54fb-49cb-aa0d-12d358360e3d" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rt2sk" Jul 05 00:21:34.623860 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:34.623826 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6bed555-54fb-49cb-aa0d-12d358360e3d" path="/var/lib/kubelet/pods/c6bed555-54fb-49cb-aa0d-12d358360e3d/volumes" Jul 05 00:21:35.622452 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:35.622420 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:35.723412 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:35.723379 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:38.210524 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:38.210491 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:38.309275 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:38.309244 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:41.415059 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:41.415018 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:43.426488 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:43.426455 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" Jul 05 00:21:59.780260 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:59.780231 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:21:59.998756 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:59.998703 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5"] Jul 05 00:21:59.998953 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:21:59.998932 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" podUID="6144ffd7-4e3f-4416-9b9a-08b81d6030d2" containerName="manager" containerID="cri-o://f0a1c0391896b2e5960dee6c94403f7eefb86868f404f6622d0c4b69a1f27a88" gracePeriod=10 Jul 05 00:22:00.254564 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.254532 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:22:00.260612 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.260591 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" Jul 05 00:22:00.283348 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.283316 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6144ffd7-4e3f-4416-9b9a-08b81d6030d2-extensions-socket-volume\") pod \"6144ffd7-4e3f-4416-9b9a-08b81d6030d2\" (UID: \"6144ffd7-4e3f-4416-9b9a-08b81d6030d2\") " Jul 05 00:22:00.283485 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.283346 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmmc9\" (UniqueName: \"kubernetes.io/projected/6144ffd7-4e3f-4416-9b9a-08b81d6030d2-kube-api-access-lmmc9\") pod \"6144ffd7-4e3f-4416-9b9a-08b81d6030d2\" (UID: \"6144ffd7-4e3f-4416-9b9a-08b81d6030d2\") " Jul 05 00:22:00.283891 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.283860 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6144ffd7-4e3f-4416-9b9a-08b81d6030d2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6144ffd7-4e3f-4416-9b9a-08b81d6030d2" (UID: "6144ffd7-4e3f-4416-9b9a-08b81d6030d2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 00:22:00.285709 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.285684 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6144ffd7-4e3f-4416-9b9a-08b81d6030d2-kube-api-access-lmmc9" (OuterVolumeSpecName: "kube-api-access-lmmc9") pod "6144ffd7-4e3f-4416-9b9a-08b81d6030d2" (UID: "6144ffd7-4e3f-4416-9b9a-08b81d6030d2"). InnerVolumeSpecName "kube-api-access-lmmc9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:22:00.384502 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.384405 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6144ffd7-4e3f-4416-9b9a-08b81d6030d2-extensions-socket-volume\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:22:00.384502 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.384439 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmmc9\" (UniqueName: \"kubernetes.io/projected/6144ffd7-4e3f-4416-9b9a-08b81d6030d2-kube-api-access-lmmc9\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:22:00.544489 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.544457 2558 generic.go:356] "Generic (PLEG): container finished" podID="6144ffd7-4e3f-4416-9b9a-08b81d6030d2" containerID="f0a1c0391896b2e5960dee6c94403f7eefb86868f404f6622d0c4b69a1f27a88" exitCode=0 Jul 05 00:22:00.544656 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.544503 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" Jul 05 00:22:00.544656 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.544527 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" event={"ID":"6144ffd7-4e3f-4416-9b9a-08b81d6030d2","Type":"ContainerDied","Data":"f0a1c0391896b2e5960dee6c94403f7eefb86868f404f6622d0c4b69a1f27a88"} Jul 05 00:22:00.544656 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.544569 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5" event={"ID":"6144ffd7-4e3f-4416-9b9a-08b81d6030d2","Type":"ContainerDied","Data":"ccf95bb56f3cbbeaa8caeb45a883d1afa814e80dccefe4feb96a1eee636d1e94"} Jul 05 00:22:00.544656 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.544589 2558 scope.go:117] "RemoveContainer" containerID="f0a1c0391896b2e5960dee6c94403f7eefb86868f404f6622d0c4b69a1f27a88" Jul 05 00:22:00.554445 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.554413 2558 scope.go:117] "RemoveContainer" containerID="f0a1c0391896b2e5960dee6c94403f7eefb86868f404f6622d0c4b69a1f27a88" Jul 05 00:22:00.554684 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:22:00.554667 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0a1c0391896b2e5960dee6c94403f7eefb86868f404f6622d0c4b69a1f27a88\": container with ID starting with f0a1c0391896b2e5960dee6c94403f7eefb86868f404f6622d0c4b69a1f27a88 not found: ID does not exist" containerID="f0a1c0391896b2e5960dee6c94403f7eefb86868f404f6622d0c4b69a1f27a88" Jul 05 00:22:00.554740 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.554700 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0a1c0391896b2e5960dee6c94403f7eefb86868f404f6622d0c4b69a1f27a88"} err="failed to get container status \"f0a1c0391896b2e5960dee6c94403f7eefb86868f404f6622d0c4b69a1f27a88\": rpc error: code = NotFound desc = could not find container \"f0a1c0391896b2e5960dee6c94403f7eefb86868f404f6622d0c4b69a1f27a88\": container with ID starting with f0a1c0391896b2e5960dee6c94403f7eefb86868f404f6622d0c4b69a1f27a88 not found: ID does not exist" Jul 05 00:22:00.568264 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.568236 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5"] Jul 05 00:22:00.571294 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.571272 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bgcj5"] Jul 05 00:22:00.623277 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:00.623253 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6144ffd7-4e3f-4416-9b9a-08b81d6030d2" path="/var/lib/kubelet/pods/6144ffd7-4e3f-4416-9b9a-08b81d6030d2/volumes" Jul 05 00:22:07.415493 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:07.415459 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:22:07.463667 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:07.463635 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:22:17.598227 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.598188 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xb746"] Jul 05 00:22:17.598729 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.598714 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6bed555-54fb-49cb-aa0d-12d358360e3d" containerName="manager" Jul 05 00:22:17.598798 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.598734 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6bed555-54fb-49cb-aa0d-12d358360e3d" containerName="manager" Jul 05 00:22:17.598798 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.598760 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6144ffd7-4e3f-4416-9b9a-08b81d6030d2" containerName="manager" Jul 05 00:22:17.598798 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.598771 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6144ffd7-4e3f-4416-9b9a-08b81d6030d2" containerName="manager" Jul 05 00:22:17.598988 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.598855 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6144ffd7-4e3f-4416-9b9a-08b81d6030d2" containerName="manager" Jul 05 00:22:17.598988 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.598870 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6bed555-54fb-49cb-aa0d-12d358360e3d" containerName="manager" Jul 05 00:22:17.601148 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.601127 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" Jul 05 00:22:17.603626 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.603602 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 05 00:22:17.603738 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.603628 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j8vgb\"" Jul 05 00:22:17.611709 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.611671 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xb746"] Jul 05 00:22:17.617818 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.617793 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:22:17.701718 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.701686 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xb746"] Jul 05 00:22:17.704006 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.703980 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:22:17.730507 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.730477 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8zvj\" (UniqueName: \"kubernetes.io/projected/2fabb0e4-b0f9-4aa1-8e78-99848fa6a415-kube-api-access-q8zvj\") pod \"limitador-limitador-64c94fd64f-xb746\" (UID: \"2fabb0e4-b0f9-4aa1-8e78-99848fa6a415\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" Jul 05 00:22:17.730647 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.730630 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2fabb0e4-b0f9-4aa1-8e78-99848fa6a415-config-file\") pod \"limitador-limitador-64c94fd64f-xb746\" (UID: \"2fabb0e4-b0f9-4aa1-8e78-99848fa6a415\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" Jul 05 00:22:17.803982 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.803895 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:22:17.832045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.832014 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8zvj\" (UniqueName: \"kubernetes.io/projected/2fabb0e4-b0f9-4aa1-8e78-99848fa6a415-kube-api-access-q8zvj\") pod \"limitador-limitador-64c94fd64f-xb746\" (UID: \"2fabb0e4-b0f9-4aa1-8e78-99848fa6a415\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" Jul 05 00:22:17.832223 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.832084 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2fabb0e4-b0f9-4aa1-8e78-99848fa6a415-config-file\") pod \"limitador-limitador-64c94fd64f-xb746\" (UID: \"2fabb0e4-b0f9-4aa1-8e78-99848fa6a415\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" Jul 05 00:22:17.832640 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.832620 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2fabb0e4-b0f9-4aa1-8e78-99848fa6a415-config-file\") pod \"limitador-limitador-64c94fd64f-xb746\" (UID: \"2fabb0e4-b0f9-4aa1-8e78-99848fa6a415\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" Jul 05 00:22:17.839729 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.839704 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8zvj\" (UniqueName: \"kubernetes.io/projected/2fabb0e4-b0f9-4aa1-8e78-99848fa6a415-kube-api-access-q8zvj\") pod \"limitador-limitador-64c94fd64f-xb746\" (UID: \"2fabb0e4-b0f9-4aa1-8e78-99848fa6a415\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" Jul 05 00:22:17.915832 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:17.915801 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" Jul 05 00:22:18.046649 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:18.046625 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xb746"] Jul 05 00:22:18.049247 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:22:18.049215 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fabb0e4_b0f9_4aa1_8e78_99848fa6a415.slice/crio-cc9249c0dfcd7995c4eb35184ec0afd0b4a160deece90d092efdc6ce7e50b072 WatchSource:0}: Error finding container cc9249c0dfcd7995c4eb35184ec0afd0b4a160deece90d092efdc6ce7e50b072: Status 404 returned error can't find the container with id cc9249c0dfcd7995c4eb35184ec0afd0b4a160deece90d092efdc6ce7e50b072 Jul 05 00:22:18.443549 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:18.443515 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:22:18.497735 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:18.497703 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:22:18.627240 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:18.627207 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" event={"ID":"2fabb0e4-b0f9-4aa1-8e78-99848fa6a415","Type":"ContainerStarted","Data":"cc9249c0dfcd7995c4eb35184ec0afd0b4a160deece90d092efdc6ce7e50b072"} Jul 05 00:22:21.643202 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:21.643163 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" event={"ID":"2fabb0e4-b0f9-4aa1-8e78-99848fa6a415","Type":"ContainerStarted","Data":"7a4df3c5d0520bffc1cdb5a996f520d58b930e3c061edee75a904e46a7f63367"} Jul 05 00:22:21.643643 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:21.643233 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" Jul 05 00:22:21.659388 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:21.659337 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" podStartSLOduration=1.901298588 podStartE2EDuration="4.659322639s" totalImagesPullingTime="2.758024051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:22:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:22:21.658032624 +0000 UTC m=+779.594398003" watchObservedRunningTime="2026-07-05 00:22:21.659322639 +0000 UTC m=+779.595688018" Jul 05 00:22:32.649941 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:32.649891 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" Jul 05 00:22:35.223004 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:35.222973 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xb746"] Jul 05 00:22:35.223446 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:35.223163 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" podUID="2fabb0e4-b0f9-4aa1-8e78-99848fa6a415" containerName="limitador" containerID="cri-o://7a4df3c5d0520bffc1cdb5a996f520d58b930e3c061edee75a904e46a7f63367" gracePeriod=30 Jul 05 00:22:35.705064 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:35.705036 2558 generic.go:356] "Generic (PLEG): container finished" podID="2fabb0e4-b0f9-4aa1-8e78-99848fa6a415" containerID="7a4df3c5d0520bffc1cdb5a996f520d58b930e3c061edee75a904e46a7f63367" exitCode=0 Jul 05 00:22:35.705188 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:35.705110 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" event={"ID":"2fabb0e4-b0f9-4aa1-8e78-99848fa6a415","Type":"ContainerDied","Data":"7a4df3c5d0520bffc1cdb5a996f520d58b930e3c061edee75a904e46a7f63367"} Jul 05 00:22:35.762413 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:35.762391 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" Jul 05 00:22:35.784191 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:35.784130 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8zvj\" (UniqueName: \"kubernetes.io/projected/2fabb0e4-b0f9-4aa1-8e78-99848fa6a415-kube-api-access-q8zvj\") pod \"2fabb0e4-b0f9-4aa1-8e78-99848fa6a415\" (UID: \"2fabb0e4-b0f9-4aa1-8e78-99848fa6a415\") " Jul 05 00:22:35.784315 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:35.784192 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2fabb0e4-b0f9-4aa1-8e78-99848fa6a415-config-file\") pod \"2fabb0e4-b0f9-4aa1-8e78-99848fa6a415\" (UID: \"2fabb0e4-b0f9-4aa1-8e78-99848fa6a415\") " Jul 05 00:22:35.784552 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:35.784517 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fabb0e4-b0f9-4aa1-8e78-99848fa6a415-config-file" (OuterVolumeSpecName: "config-file") pod "2fabb0e4-b0f9-4aa1-8e78-99848fa6a415" (UID: "2fabb0e4-b0f9-4aa1-8e78-99848fa6a415"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:22:35.786347 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:35.786329 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fabb0e4-b0f9-4aa1-8e78-99848fa6a415-kube-api-access-q8zvj" (OuterVolumeSpecName: "kube-api-access-q8zvj") pod "2fabb0e4-b0f9-4aa1-8e78-99848fa6a415" (UID: "2fabb0e4-b0f9-4aa1-8e78-99848fa6a415"). InnerVolumeSpecName "kube-api-access-q8zvj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:22:35.884802 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:35.884769 2558 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2fabb0e4-b0f9-4aa1-8e78-99848fa6a415-config-file\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:22:35.884802 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:35.884798 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8zvj\" (UniqueName: \"kubernetes.io/projected/2fabb0e4-b0f9-4aa1-8e78-99848fa6a415-kube-api-access-q8zvj\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:22:36.710526 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:36.710502 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" Jul 05 00:22:36.710526 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:36.710518 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xb746" event={"ID":"2fabb0e4-b0f9-4aa1-8e78-99848fa6a415","Type":"ContainerDied","Data":"cc9249c0dfcd7995c4eb35184ec0afd0b4a160deece90d092efdc6ce7e50b072"} Jul 05 00:22:36.710989 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:36.710577 2558 scope.go:117] "RemoveContainer" containerID="7a4df3c5d0520bffc1cdb5a996f520d58b930e3c061edee75a904e46a7f63367" Jul 05 00:22:36.726652 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:36.726630 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xb746"] Jul 05 00:22:36.730446 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:36.730426 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xb746"] Jul 05 00:22:38.623579 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:38.623546 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fabb0e4-b0f9-4aa1-8e78-99848fa6a415" path="/var/lib/kubelet/pods/2fabb0e4-b0f9-4aa1-8e78-99848fa6a415/volumes" Jul 05 00:22:41.010763 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.010733 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-8cpxr"] Jul 05 00:22:41.011135 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.011103 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fabb0e4-b0f9-4aa1-8e78-99848fa6a415" containerName="limitador" Jul 05 00:22:41.011135 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.011115 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fabb0e4-b0f9-4aa1-8e78-99848fa6a415" containerName="limitador" Jul 05 00:22:41.011205 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.011174 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fabb0e4-b0f9-4aa1-8e78-99848fa6a415" containerName="limitador" Jul 05 00:22:41.013810 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.013790 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-8cpxr" Jul 05 00:22:41.016372 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.016352 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 05 00:22:41.016480 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.016397 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mdx7s\"" Jul 05 00:22:41.022957 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.022932 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-8cpxr"] Jul 05 00:22:41.029463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.029437 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-456b8\" (UniqueName: \"kubernetes.io/projected/6da95d6e-6b88-4f8b-9472-f44ab80723cc-kube-api-access-456b8\") pod \"postgres-69fb4594cf-8cpxr\" (UID: \"6da95d6e-6b88-4f8b-9472-f44ab80723cc\") " pod="opendatahub/postgres-69fb4594cf-8cpxr" Jul 05 00:22:41.029574 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.029470 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6da95d6e-6b88-4f8b-9472-f44ab80723cc-data\") pod \"postgres-69fb4594cf-8cpxr\" (UID: \"6da95d6e-6b88-4f8b-9472-f44ab80723cc\") " pod="opendatahub/postgres-69fb4594cf-8cpxr" Jul 05 00:22:41.105982 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.105947 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:22:41.129989 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.129952 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-456b8\" (UniqueName: \"kubernetes.io/projected/6da95d6e-6b88-4f8b-9472-f44ab80723cc-kube-api-access-456b8\") pod \"postgres-69fb4594cf-8cpxr\" (UID: \"6da95d6e-6b88-4f8b-9472-f44ab80723cc\") " pod="opendatahub/postgres-69fb4594cf-8cpxr" Jul 05 00:22:41.129989 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.129987 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6da95d6e-6b88-4f8b-9472-f44ab80723cc-data\") pod \"postgres-69fb4594cf-8cpxr\" (UID: \"6da95d6e-6b88-4f8b-9472-f44ab80723cc\") " pod="opendatahub/postgres-69fb4594cf-8cpxr" Jul 05 00:22:41.130360 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.130343 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6da95d6e-6b88-4f8b-9472-f44ab80723cc-data\") pod \"postgres-69fb4594cf-8cpxr\" (UID: \"6da95d6e-6b88-4f8b-9472-f44ab80723cc\") " pod="opendatahub/postgres-69fb4594cf-8cpxr" Jul 05 00:22:41.137770 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.137739 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-456b8\" (UniqueName: \"kubernetes.io/projected/6da95d6e-6b88-4f8b-9472-f44ab80723cc-kube-api-access-456b8\") pod \"postgres-69fb4594cf-8cpxr\" (UID: \"6da95d6e-6b88-4f8b-9472-f44ab80723cc\") " pod="opendatahub/postgres-69fb4594cf-8cpxr" Jul 05 00:22:41.325297 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.325218 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-8cpxr" Jul 05 00:22:41.454800 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.454768 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-8cpxr"] Jul 05 00:22:41.458403 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:22:41.458375 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da95d6e_6b88_4f8b_9472_f44ab80723cc.slice/crio-eee9b1d6238825b36c6921d70758c706a34dec39101ebe2f24a153df3cef6ffc WatchSource:0}: Error finding container eee9b1d6238825b36c6921d70758c706a34dec39101ebe2f24a153df3cef6ffc: Status 404 returned error can't find the container with id eee9b1d6238825b36c6921d70758c706a34dec39101ebe2f24a153df3cef6ffc Jul 05 00:22:41.733766 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:41.733732 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-8cpxr" event={"ID":"6da95d6e-6b88-4f8b-9472-f44ab80723cc","Type":"ContainerStarted","Data":"eee9b1d6238825b36c6921d70758c706a34dec39101ebe2f24a153df3cef6ffc"} Jul 05 00:22:47.583592 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:47.583560 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:22:47.766001 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:47.765965 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-8cpxr" event={"ID":"6da95d6e-6b88-4f8b-9472-f44ab80723cc","Type":"ContainerStarted","Data":"65b7ad7c45c5dcbbfac780b57a48796fb4b13f44a5ca7d2eebac6752f4c94505"} Jul 05 00:22:47.766223 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:47.766090 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-8cpxr" Jul 05 00:22:47.787974 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:47.787896 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-8cpxr" podStartSLOduration=1.9026074099999999 podStartE2EDuration="7.787882538s" totalImagesPullingTime="5.885275128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:22:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:22:47.786123779 +0000 UTC m=+805.722489177" watchObservedRunningTime="2026-07-05 00:22:47.787882538 +0000 UTC m=+805.724247917" Jul 05 00:22:53.801730 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:53.801703 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-8cpxr" Jul 05 00:22:54.113679 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:54.113586 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:22:54.453317 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:22:54.453281 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:00.623669 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.623635 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64c9674b84-dg6st"] Jul 05 00:23:00.629613 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.629581 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:00.642369 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.642343 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64c9674b84-dg6st"] Jul 05 00:23:00.642501 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.642441 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:00.645143 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.645121 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 05 00:23:00.645280 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.645150 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 05 00:23:00.645280 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.645182 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ngppp\"" Jul 05 00:23:00.697835 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.697808 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:00.701043 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.701026 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-webhook-cert\") pod \"maas-controller-64c9674b84-dg6st\" (UID: \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\") " pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:00.701141 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.701060 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbsb4\" (UniqueName: \"kubernetes.io/projected/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-kube-api-access-gbsb4\") pod \"maas-controller-64c9674b84-dg6st\" (UID: \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\") " pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:00.701141 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.701086 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-service-ca\") pod \"maas-controller-64c9674b84-dg6st\" (UID: \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\") " pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:00.793638 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.793604 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:00.794085 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.794065 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-98468dc4b-gcwk5"] Jul 05 00:23:00.796422 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.796406 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:23:00.801599 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.801579 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-webhook-cert\") pod \"maas-controller-64c9674b84-dg6st\" (UID: \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\") " pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:00.801709 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.801621 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbsb4\" (UniqueName: \"kubernetes.io/projected/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-kube-api-access-gbsb4\") pod \"maas-controller-64c9674b84-dg6st\" (UID: \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\") " pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:00.801709 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.801664 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-service-ca\") pod \"maas-controller-64c9674b84-dg6st\" (UID: \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\") " pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:00.802477 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.802451 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-service-ca\") pod \"maas-controller-64c9674b84-dg6st\" (UID: \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\") " pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:00.804846 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.804825 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-webhook-cert\") pod \"maas-controller-64c9674b84-dg6st\" (UID: \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\") " pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:00.805947 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.805928 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-98468dc4b-gcwk5"] Jul 05 00:23:00.811708 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.811689 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbsb4\" (UniqueName: \"kubernetes.io/projected/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-kube-api-access-gbsb4\") pod \"maas-controller-64c9674b84-dg6st\" (UID: \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\") " pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:00.850656 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.850627 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:00.903149 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.903120 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/061ca742-3d48-4d61-9d27-51ce74c41af3-webhook-cert\") pod \"maas-controller-98468dc4b-gcwk5\" (UID: \"061ca742-3d48-4d61-9d27-51ce74c41af3\") " pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:23:00.903293 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.903160 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/061ca742-3d48-4d61-9d27-51ce74c41af3-service-ca\") pod \"maas-controller-98468dc4b-gcwk5\" (UID: \"061ca742-3d48-4d61-9d27-51ce74c41af3\") " pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:23:00.903293 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.903191 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tqgz\" (UniqueName: \"kubernetes.io/projected/061ca742-3d48-4d61-9d27-51ce74c41af3-kube-api-access-7tqgz\") pod \"maas-controller-98468dc4b-gcwk5\" (UID: \"061ca742-3d48-4d61-9d27-51ce74c41af3\") " pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:23:00.952921 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:00.952874 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:01.003996 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:01.003959 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/061ca742-3d48-4d61-9d27-51ce74c41af3-service-ca\") pod \"maas-controller-98468dc4b-gcwk5\" (UID: \"061ca742-3d48-4d61-9d27-51ce74c41af3\") " pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:23:01.004467 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:01.004033 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tqgz\" (UniqueName: \"kubernetes.io/projected/061ca742-3d48-4d61-9d27-51ce74c41af3-kube-api-access-7tqgz\") pod \"maas-controller-98468dc4b-gcwk5\" (UID: \"061ca742-3d48-4d61-9d27-51ce74c41af3\") " pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:23:01.004467 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:01.004124 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/061ca742-3d48-4d61-9d27-51ce74c41af3-webhook-cert\") pod \"maas-controller-98468dc4b-gcwk5\" (UID: \"061ca742-3d48-4d61-9d27-51ce74c41af3\") " pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:23:01.005169 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:01.005002 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/061ca742-3d48-4d61-9d27-51ce74c41af3-service-ca\") pod \"maas-controller-98468dc4b-gcwk5\" (UID: \"061ca742-3d48-4d61-9d27-51ce74c41af3\") " pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:23:01.006895 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:01.006855 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/061ca742-3d48-4d61-9d27-51ce74c41af3-webhook-cert\") pod \"maas-controller-98468dc4b-gcwk5\" (UID: \"061ca742-3d48-4d61-9d27-51ce74c41af3\") " pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:23:01.012577 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:01.012548 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tqgz\" (UniqueName: \"kubernetes.io/projected/061ca742-3d48-4d61-9d27-51ce74c41af3-kube-api-access-7tqgz\") pod \"maas-controller-98468dc4b-gcwk5\" (UID: \"061ca742-3d48-4d61-9d27-51ce74c41af3\") " pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:23:01.087800 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:01.087771 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64c9674b84-dg6st"] Jul 05 00:23:01.091217 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:23:01.091190 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf049a552_04ce_40b6_8bc5_b0f7b195f3c5.slice/crio-c3fa11989746a76b845422f42f7966589c77700d174b7d7365cd86e59bbdeb47 WatchSource:0}: Error finding container c3fa11989746a76b845422f42f7966589c77700d174b7d7365cd86e59bbdeb47: Status 404 returned error can't find the container with id c3fa11989746a76b845422f42f7966589c77700d174b7d7365cd86e59bbdeb47 Jul 05 00:23:01.107069 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:01.107040 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:23:01.236497 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:01.236351 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-98468dc4b-gcwk5"] Jul 05 00:23:01.239035 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:23:01.239014 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod061ca742_3d48_4d61_9d27_51ce74c41af3.slice/crio-b861995839e0de3217b248d2f760096f1c10b5507cff166c5e501269146e93c0 WatchSource:0}: Error finding container b861995839e0de3217b248d2f760096f1c10b5507cff166c5e501269146e93c0: Status 404 returned error can't find the container with id b861995839e0de3217b248d2f760096f1c10b5507cff166c5e501269146e93c0 Jul 05 00:23:01.831309 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:01.831219 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64c9674b84-dg6st" event={"ID":"f049a552-04ce-40b6-8bc5-b0f7b195f3c5","Type":"ContainerStarted","Data":"c3fa11989746a76b845422f42f7966589c77700d174b7d7365cd86e59bbdeb47"} Jul 05 00:23:01.833665 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:01.833633 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98468dc4b-gcwk5" event={"ID":"061ca742-3d48-4d61-9d27-51ce74c41af3","Type":"ContainerStarted","Data":"b861995839e0de3217b248d2f760096f1c10b5507cff166c5e501269146e93c0"} Jul 05 00:23:03.451183 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:03.451152 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:03.550509 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:03.550435 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:03.844072 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:03.843978 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98468dc4b-gcwk5" event={"ID":"061ca742-3d48-4d61-9d27-51ce74c41af3","Type":"ContainerStarted","Data":"619dcb8f97c7ec45d75804cd14673b0e114021c322f1a4df5e76ec3f37879399"} Jul 05 00:23:03.844217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:03.844083 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:23:03.845371 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:03.845352 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64c9674b84-dg6st" event={"ID":"f049a552-04ce-40b6-8bc5-b0f7b195f3c5","Type":"ContainerStarted","Data":"08d95e03a9f8f51d6484b8230b1a4e0c859fbe9adeff5e25ffc2558edcfe575a"} Jul 05 00:23:03.845505 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:03.845490 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:03.861856 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:03.861800 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-98468dc4b-gcwk5" podStartSLOduration=1.8395175350000001 podStartE2EDuration="3.861782534s" totalImagesPullingTime="2.022264999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:23:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:23:03.859753506 +0000 UTC m=+821.796118885" watchObservedRunningTime="2026-07-05 00:23:03.861782534 +0000 UTC m=+821.798147914" Jul 05 00:23:03.876268 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:03.876222 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64c9674b84-dg6st" podStartSLOduration=1.695785373 podStartE2EDuration="3.876208868s" totalImagesPullingTime="2.180423495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:23:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:23:03.874453557 +0000 UTC m=+821.810818937" watchObservedRunningTime="2026-07-05 00:23:03.876208868 +0000 UTC m=+821.812574268" Jul 05 00:23:14.856248 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:14.856221 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:23:14.856761 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:14.856599 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:14.918243 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:14.918207 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64c9674b84-dg6st"] Jul 05 00:23:14.918503 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:14.918468 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-64c9674b84-dg6st" podUID="f049a552-04ce-40b6-8bc5-b0f7b195f3c5" containerName="manager" containerID="cri-o://08d95e03a9f8f51d6484b8230b1a4e0c859fbe9adeff5e25ffc2558edcfe575a" gracePeriod=30 Jul 05 00:23:15.162666 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.162643 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:15.332723 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.332688 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbsb4\" (UniqueName: \"kubernetes.io/projected/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-kube-api-access-gbsb4\") pod \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\" (UID: \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\") " Jul 05 00:23:15.332930 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.332741 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-webhook-cert\") pod \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\" (UID: \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\") " Jul 05 00:23:15.333014 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.332969 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-service-ca\") pod \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\" (UID: \"f049a552-04ce-40b6-8bc5-b0f7b195f3c5\") " Jul 05 00:23:15.333268 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.333236 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-service-ca" (OuterVolumeSpecName: "service-ca") pod "f049a552-04ce-40b6-8bc5-b0f7b195f3c5" (UID: "f049a552-04ce-40b6-8bc5-b0f7b195f3c5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:23:15.335100 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.335074 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-kube-api-access-gbsb4" (OuterVolumeSpecName: "kube-api-access-gbsb4") pod "f049a552-04ce-40b6-8bc5-b0f7b195f3c5" (UID: "f049a552-04ce-40b6-8bc5-b0f7b195f3c5"). InnerVolumeSpecName "kube-api-access-gbsb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:23:15.335198 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.335117 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f049a552-04ce-40b6-8bc5-b0f7b195f3c5" (UID: "f049a552-04ce-40b6-8bc5-b0f7b195f3c5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 00:23:15.434483 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.434395 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbsb4\" (UniqueName: \"kubernetes.io/projected/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-kube-api-access-gbsb4\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:23:15.434483 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.434437 2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-webhook-cert\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:23:15.434483 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.434446 2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f049a552-04ce-40b6-8bc5-b0f7b195f3c5-service-ca\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:23:15.900197 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.900159 2558 generic.go:356] "Generic (PLEG): container finished" podID="f049a552-04ce-40b6-8bc5-b0f7b195f3c5" containerID="08d95e03a9f8f51d6484b8230b1a4e0c859fbe9adeff5e25ffc2558edcfe575a" exitCode=0 Jul 05 00:23:15.900624 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.900208 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64c9674b84-dg6st" Jul 05 00:23:15.900624 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.900252 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64c9674b84-dg6st" event={"ID":"f049a552-04ce-40b6-8bc5-b0f7b195f3c5","Type":"ContainerDied","Data":"08d95e03a9f8f51d6484b8230b1a4e0c859fbe9adeff5e25ffc2558edcfe575a"} Jul 05 00:23:15.900624 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.900297 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64c9674b84-dg6st" event={"ID":"f049a552-04ce-40b6-8bc5-b0f7b195f3c5","Type":"ContainerDied","Data":"c3fa11989746a76b845422f42f7966589c77700d174b7d7365cd86e59bbdeb47"} Jul 05 00:23:15.900624 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.900320 2558 scope.go:117] "RemoveContainer" containerID="08d95e03a9f8f51d6484b8230b1a4e0c859fbe9adeff5e25ffc2558edcfe575a" Jul 05 00:23:15.908806 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.908786 2558 scope.go:117] "RemoveContainer" containerID="08d95e03a9f8f51d6484b8230b1a4e0c859fbe9adeff5e25ffc2558edcfe575a" Jul 05 00:23:15.909121 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:23:15.909082 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08d95e03a9f8f51d6484b8230b1a4e0c859fbe9adeff5e25ffc2558edcfe575a\": container with ID starting with 08d95e03a9f8f51d6484b8230b1a4e0c859fbe9adeff5e25ffc2558edcfe575a not found: ID does not exist" containerID="08d95e03a9f8f51d6484b8230b1a4e0c859fbe9adeff5e25ffc2558edcfe575a" Jul 05 00:23:15.909229 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.909121 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08d95e03a9f8f51d6484b8230b1a4e0c859fbe9adeff5e25ffc2558edcfe575a"} err="failed to get container status \"08d95e03a9f8f51d6484b8230b1a4e0c859fbe9adeff5e25ffc2558edcfe575a\": rpc error: code = NotFound desc = could not find container \"08d95e03a9f8f51d6484b8230b1a4e0c859fbe9adeff5e25ffc2558edcfe575a\": container with ID starting with 08d95e03a9f8f51d6484b8230b1a4e0c859fbe9adeff5e25ffc2558edcfe575a not found: ID does not exist" Jul 05 00:23:15.920190 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.920160 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64c9674b84-dg6st"] Jul 05 00:23:15.921795 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:15.921777 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64c9674b84-dg6st"] Jul 05 00:23:16.562528 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:16.562493 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:16.616317 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:16.616291 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:16.624036 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:16.624011 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f049a552-04ce-40b6-8bc5-b0f7b195f3c5" path="/var/lib/kubelet/pods/f049a552-04ce-40b6-8bc5-b0f7b195f3c5/volumes" Jul 05 00:23:34.532008 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:34.531976 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:34.956628 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:34.956597 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:34.990804 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:34.990776 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5b96d4fbd4-5b8hr"] Jul 05 00:23:34.991160 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:34.991147 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f049a552-04ce-40b6-8bc5-b0f7b195f3c5" containerName="manager" Jul 05 00:23:34.991208 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:34.991163 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f049a552-04ce-40b6-8bc5-b0f7b195f3c5" containerName="manager" Jul 05 00:23:34.991241 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:34.991228 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f049a552-04ce-40b6-8bc5-b0f7b195f3c5" containerName="manager" Jul 05 00:23:34.995620 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:34.995603 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" Jul 05 00:23:34.998330 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:34.998311 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2p4nn\"" Jul 05 00:23:34.999354 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:34.999336 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 05 00:23:35.003974 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.003951 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5b96d4fbd4-5b8hr"] Jul 05 00:23:35.056437 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.056403 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:35.084653 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.084618 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f82e5b-362d-485e-980e-23891749ff79-openshift-service-ca\") pod \"maas-api-5b96d4fbd4-5b8hr\" (UID: \"c1f82e5b-362d-485e-980e-23891749ff79\") " pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" Jul 05 00:23:35.084840 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.084740 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj48s\" (UniqueName: \"kubernetes.io/projected/c1f82e5b-362d-485e-980e-23891749ff79-kube-api-access-xj48s\") pod \"maas-api-5b96d4fbd4-5b8hr\" (UID: \"c1f82e5b-362d-485e-980e-23891749ff79\") " pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" Jul 05 00:23:35.084840 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.084776 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1f82e5b-362d-485e-980e-23891749ff79-maas-api-tls\") pod \"maas-api-5b96d4fbd4-5b8hr\" (UID: \"c1f82e5b-362d-485e-980e-23891749ff79\") " pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" Jul 05 00:23:35.186046 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.186010 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj48s\" (UniqueName: \"kubernetes.io/projected/c1f82e5b-362d-485e-980e-23891749ff79-kube-api-access-xj48s\") pod \"maas-api-5b96d4fbd4-5b8hr\" (UID: \"c1f82e5b-362d-485e-980e-23891749ff79\") " pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" Jul 05 00:23:35.186244 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.186054 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1f82e5b-362d-485e-980e-23891749ff79-maas-api-tls\") pod \"maas-api-5b96d4fbd4-5b8hr\" (UID: \"c1f82e5b-362d-485e-980e-23891749ff79\") " pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" Jul 05 00:23:35.186244 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.186083 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f82e5b-362d-485e-980e-23891749ff79-openshift-service-ca\") pod \"maas-api-5b96d4fbd4-5b8hr\" (UID: \"c1f82e5b-362d-485e-980e-23891749ff79\") " pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" Jul 05 00:23:35.186868 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.186838 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f82e5b-362d-485e-980e-23891749ff79-openshift-service-ca\") pod \"maas-api-5b96d4fbd4-5b8hr\" (UID: \"c1f82e5b-362d-485e-980e-23891749ff79\") " pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" Jul 05 00:23:35.188879 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.188853 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1f82e5b-362d-485e-980e-23891749ff79-maas-api-tls\") pod \"maas-api-5b96d4fbd4-5b8hr\" (UID: \"c1f82e5b-362d-485e-980e-23891749ff79\") " pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" Jul 05 00:23:35.194016 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.193996 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj48s\" (UniqueName: \"kubernetes.io/projected/c1f82e5b-362d-485e-980e-23891749ff79-kube-api-access-xj48s\") pod \"maas-api-5b96d4fbd4-5b8hr\" (UID: \"c1f82e5b-362d-485e-980e-23891749ff79\") " pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" Jul 05 00:23:35.308220 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.308136 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" Jul 05 00:23:35.349561 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.349523 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:35.448998 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.448971 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5b96d4fbd4-5b8hr"] Jul 05 00:23:35.450827 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.450800 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:35.451485 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:23:35.451465 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f82e5b_362d_485e_980e_23891749ff79.slice/crio-afefbee5f17490eaf4c26cec19ccb3c5c0f5bcf2ef788a39659593babfb3a8a6 WatchSource:0}: Error finding container afefbee5f17490eaf4c26cec19ccb3c5c0f5bcf2ef788a39659593babfb3a8a6: Status 404 returned error can't find the container with id afefbee5f17490eaf4c26cec19ccb3c5c0f5bcf2ef788a39659593babfb3a8a6 Jul 05 00:23:35.454823 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.454799 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-q5zsp"] Jul 05 00:23:35.459220 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.459199 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-q5zsp" Jul 05 00:23:35.461762 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.461745 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-kcdlt\"" Jul 05 00:23:35.461844 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.461745 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 05 00:23:35.466523 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.466497 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-q5zsp"] Jul 05 00:23:35.481953 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.481923 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-mmxgl"] Jul 05 00:23:35.485139 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.485118 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-mmxgl" Jul 05 00:23:35.488630 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.488603 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztkw7\" (UniqueName: \"kubernetes.io/projected/99d67b88-480f-48fd-9e06-07d9b217d717-kube-api-access-ztkw7\") pod \"payload-pre-processing-7964457696-q5zsp\" (UID: \"99d67b88-480f-48fd-9e06-07d9b217d717\") " pod="openshift-ingress/payload-pre-processing-7964457696-q5zsp" Jul 05 00:23:35.488731 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.488653 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99d67b88-480f-48fd-9e06-07d9b217d717-plugins-config-volume\") pod \"payload-pre-processing-7964457696-q5zsp\" (UID: \"99d67b88-480f-48fd-9e06-07d9b217d717\") " pod="openshift-ingress/payload-pre-processing-7964457696-q5zsp" Jul 05 00:23:35.494355 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.494333 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-mmxgl"] Jul 05 00:23:35.589859 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.589776 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztkw7\" (UniqueName: \"kubernetes.io/projected/99d67b88-480f-48fd-9e06-07d9b217d717-kube-api-access-ztkw7\") pod \"payload-pre-processing-7964457696-q5zsp\" (UID: \"99d67b88-480f-48fd-9e06-07d9b217d717\") " pod="openshift-ingress/payload-pre-processing-7964457696-q5zsp" Jul 05 00:23:35.589859 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.589819 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/025bd7ca-0892-4e06-a7c7-764a46975c78-plugins-config-volume\") pod \"payload-processing-58f86c7798-mmxgl\" (UID: \"025bd7ca-0892-4e06-a7c7-764a46975c78\") " pod="openshift-ingress/payload-processing-58f86c7798-mmxgl" Jul 05 00:23:35.590390 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.589883 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99d67b88-480f-48fd-9e06-07d9b217d717-plugins-config-volume\") pod \"payload-pre-processing-7964457696-q5zsp\" (UID: \"99d67b88-480f-48fd-9e06-07d9b217d717\") " pod="openshift-ingress/payload-pre-processing-7964457696-q5zsp" Jul 05 00:23:35.590390 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.589966 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbwm7\" (UniqueName: \"kubernetes.io/projected/025bd7ca-0892-4e06-a7c7-764a46975c78-kube-api-access-qbwm7\") pod \"payload-processing-58f86c7798-mmxgl\" (UID: \"025bd7ca-0892-4e06-a7c7-764a46975c78\") " pod="openshift-ingress/payload-processing-58f86c7798-mmxgl" Jul 05 00:23:35.590748 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.590724 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99d67b88-480f-48fd-9e06-07d9b217d717-plugins-config-volume\") pod \"payload-pre-processing-7964457696-q5zsp\" (UID: \"99d67b88-480f-48fd-9e06-07d9b217d717\") " pod="openshift-ingress/payload-pre-processing-7964457696-q5zsp" Jul 05 00:23:35.597596 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.597575 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztkw7\" (UniqueName: \"kubernetes.io/projected/99d67b88-480f-48fd-9e06-07d9b217d717-kube-api-access-ztkw7\") pod \"payload-pre-processing-7964457696-q5zsp\" (UID: \"99d67b88-480f-48fd-9e06-07d9b217d717\") " pod="openshift-ingress/payload-pre-processing-7964457696-q5zsp" Jul 05 00:23:35.691323 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.691288 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/025bd7ca-0892-4e06-a7c7-764a46975c78-plugins-config-volume\") pod \"payload-processing-58f86c7798-mmxgl\" (UID: \"025bd7ca-0892-4e06-a7c7-764a46975c78\") " pod="openshift-ingress/payload-processing-58f86c7798-mmxgl" Jul 05 00:23:35.691510 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.691428 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbwm7\" (UniqueName: \"kubernetes.io/projected/025bd7ca-0892-4e06-a7c7-764a46975c78-kube-api-access-qbwm7\") pod \"payload-processing-58f86c7798-mmxgl\" (UID: \"025bd7ca-0892-4e06-a7c7-764a46975c78\") " pod="openshift-ingress/payload-processing-58f86c7798-mmxgl" Jul 05 00:23:35.692103 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.692077 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/025bd7ca-0892-4e06-a7c7-764a46975c78-plugins-config-volume\") pod \"payload-processing-58f86c7798-mmxgl\" (UID: \"025bd7ca-0892-4e06-a7c7-764a46975c78\") " pod="openshift-ingress/payload-processing-58f86c7798-mmxgl" Jul 05 00:23:35.699166 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.699147 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbwm7\" (UniqueName: \"kubernetes.io/projected/025bd7ca-0892-4e06-a7c7-764a46975c78-kube-api-access-qbwm7\") pod \"payload-processing-58f86c7798-mmxgl\" (UID: \"025bd7ca-0892-4e06-a7c7-764a46975c78\") " pod="openshift-ingress/payload-processing-58f86c7798-mmxgl" Jul 05 00:23:35.771535 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.771504 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-q5zsp" Jul 05 00:23:35.795836 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.795808 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-mmxgl" Jul 05 00:23:35.918322 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.918287 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-q5zsp"] Jul 05 00:23:35.922000 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:23:35.921961 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d67b88_480f_48fd_9e06_07d9b217d717.slice/crio-72c40ea209698cdadd4b7921d53e5a783ba43f1329ed908efc47ea59ac7c8601 WatchSource:0}: Error finding container 72c40ea209698cdadd4b7921d53e5a783ba43f1329ed908efc47ea59ac7c8601: Status 404 returned error can't find the container with id 72c40ea209698cdadd4b7921d53e5a783ba43f1329ed908efc47ea59ac7c8601 Jul 05 00:23:35.949610 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.949587 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-mmxgl"] Jul 05 00:23:35.951560 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:23:35.951536 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod025bd7ca_0892_4e06_a7c7_764a46975c78.slice/crio-9ee38841d7d0ffc2fa30ab0a39784f338aaadab82284572290017b26bdd759b3 WatchSource:0}: Error finding container 9ee38841d7d0ffc2fa30ab0a39784f338aaadab82284572290017b26bdd759b3: Status 404 returned error can't find the container with id 9ee38841d7d0ffc2fa30ab0a39784f338aaadab82284572290017b26bdd759b3 Jul 05 00:23:35.995621 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.995583 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-mmxgl" event={"ID":"025bd7ca-0892-4e06-a7c7-764a46975c78","Type":"ContainerStarted","Data":"9ee38841d7d0ffc2fa30ab0a39784f338aaadab82284572290017b26bdd759b3"} Jul 05 00:23:35.996781 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.996751 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-q5zsp" event={"ID":"99d67b88-480f-48fd-9e06-07d9b217d717","Type":"ContainerStarted","Data":"72c40ea209698cdadd4b7921d53e5a783ba43f1329ed908efc47ea59ac7c8601"} Jul 05 00:23:35.997853 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:35.997832 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" event={"ID":"c1f82e5b-362d-485e-980e-23891749ff79","Type":"ContainerStarted","Data":"afefbee5f17490eaf4c26cec19ccb3c5c0f5bcf2ef788a39659593babfb3a8a6"} Jul 05 00:23:38.015922 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:38.015870 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-mmxgl" event={"ID":"025bd7ca-0892-4e06-a7c7-764a46975c78","Type":"ContainerStarted","Data":"046890c1d9f37b071acc17ece136283014b88b915f0000af34a50a91b60cc6dd"} Jul 05 00:23:38.016335 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:38.015993 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-mmxgl" Jul 05 00:23:38.017446 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:38.017421 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-q5zsp" event={"ID":"99d67b88-480f-48fd-9e06-07d9b217d717","Type":"ContainerStarted","Data":"05a1907d2634d3475eb1662cc4ed4cd9e541038d851a623901639d5050e65569"} Jul 05 00:23:38.017567 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:38.017471 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-q5zsp" Jul 05 00:23:38.018809 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:38.018789 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" event={"ID":"c1f82e5b-362d-485e-980e-23891749ff79","Type":"ContainerStarted","Data":"c97a180077381957288bc0eb0fb2c9fa64f360c6c6519cd857216b76e4ef1381"} Jul 05 00:23:38.019013 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:38.018994 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" Jul 05 00:23:38.032364 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:38.032325 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-mmxgl" podStartSLOduration=1.486230004 podStartE2EDuration="3.03231198s" totalImagesPullingTime="1.546081976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:23:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:23:38.031004484 +0000 UTC m=+855.967369864" watchObservedRunningTime="2026-07-05 00:23:38.03231198 +0000 UTC m=+855.968677358" Jul 05 00:23:38.046389 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:38.046337 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" podStartSLOduration=2.475083489 podStartE2EDuration="4.046324275s" totalImagesPullingTime="1.571240786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:23:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:23:38.045230706 +0000 UTC m=+855.981596076" watchObservedRunningTime="2026-07-05 00:23:38.046324275 +0000 UTC m=+855.982689657" Jul 05 00:23:38.062147 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:38.062096 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-q5zsp" podStartSLOduration=1.4891181 podStartE2EDuration="3.062077534s" totalImagesPullingTime="1.572959434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:23:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:23:38.059779599 +0000 UTC m=+855.996144980" watchObservedRunningTime="2026-07-05 00:23:38.062077534 +0000 UTC m=+855.998442918" Jul 05 00:23:44.028692 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:44.028660 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5b96d4fbd4-5b8hr" Jul 05 00:23:45.000963 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:45.000885 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:45.101029 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:45.100996 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:49.025715 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:49.025688 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-mmxgl" Jul 05 00:23:49.026244 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:49.025836 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-q5zsp" Jul 05 00:23:52.102341 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.102312 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp"] Jul 05 00:23:52.105413 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.105395 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.108146 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.108121 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 05 00:23:52.109333 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.109311 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 05 00:23:52.109466 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.109443 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vz78f\"" Jul 05 00:23:52.109528 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.109466 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 05 00:23:52.115094 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.115072 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp"] Jul 05 00:23:52.131705 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.131677 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:23:52.238042 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.238006 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.238223 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.238048 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45d62\" (UniqueName: \"kubernetes.io/projected/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-kube-api-access-45d62\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.238223 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.238074 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.238223 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.238140 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.238223 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.238174 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.238223 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.238200 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.339342 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.339301 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.339532 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.339359 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.339532 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.339386 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.339532 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.339407 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.339532 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.339450 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45d62\" (UniqueName: \"kubernetes.io/projected/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-kube-api-access-45d62\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.339532 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.339486 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.339839 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.339764 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.339897 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.339851 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.339966 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.339895 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.341866 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.341847 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.341972 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.341958 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.347827 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.347796 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45d62\" (UniqueName: \"kubernetes.io/projected/ac6934f5-58c6-45ea-9a7b-7a7b7023bda1-kube-api-access-45d62\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp\" (UID: \"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.417223 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.417193 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:23:52.555386 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:52.555352 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp"] Jul 05 00:23:52.558173 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:23:52.558148 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6934f5_58c6_45ea_9a7b_7a7b7023bda1.slice/crio-40ab351bb76850f8103ab10794e39ec917ca9517849e555471c047d6f7c07729 WatchSource:0}: Error finding container 40ab351bb76850f8103ab10794e39ec917ca9517849e555471c047d6f7c07729: Status 404 returned error can't find the container with id 40ab351bb76850f8103ab10794e39ec917ca9517849e555471c047d6f7c07729 Jul 05 00:23:53.096674 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:53.096639 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" event={"ID":"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1","Type":"ContainerStarted","Data":"40ab351bb76850f8103ab10794e39ec917ca9517849e555471c047d6f7c07729"} Jul 05 00:23:54.102606 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:54.102514 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" event={"ID":"ac6934f5-58c6-45ea-9a7b-7a7b7023bda1","Type":"ContainerStarted","Data":"237f1baa81ee9b7e308183065e0622a191901ed5f8373037587a3ce5e19a8f05"} Jul 05 00:23:54.120714 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:23:54.120666 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" podStartSLOduration=1.032950087 podStartE2EDuration="2.120650859s" totalImagesPullingTime="1.087700772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:23:54.118267526 +0000 UTC m=+872.054632925" watchObservedRunningTime="2026-07-05 00:23:54.120650859 +0000 UTC m=+872.057016237" Jul 05 00:24:02.403809 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.403768 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g"] Jul 05 00:24:02.407823 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.407788 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.410363 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.410344 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 05 00:24:02.416546 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.416523 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g"] Jul 05 00:24:02.417332 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.417315 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:24:02.417436 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.417370 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:24:02.428745 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.428715 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:24:02.430536 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.430519 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:24:02.533923 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.533876 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4951f64-509d-4d21-827c-602b6d8d904d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.534105 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.533931 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4951f64-509d-4d21-827c-602b6d8d904d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.534105 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.534045 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4v49\" (UniqueName: \"kubernetes.io/projected/a4951f64-509d-4d21-827c-602b6d8d904d-kube-api-access-m4v49\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.534218 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.534108 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4951f64-509d-4d21-827c-602b6d8d904d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.534218 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.534132 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4951f64-509d-4d21-827c-602b6d8d904d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.534320 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.534242 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4951f64-509d-4d21-827c-602b6d8d904d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.635013 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.634981 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4951f64-509d-4d21-827c-602b6d8d904d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.635199 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.635024 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4951f64-509d-4d21-827c-602b6d8d904d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.635199 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.635049 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4951f64-509d-4d21-827c-602b6d8d904d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.635199 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.635111 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4v49\" (UniqueName: \"kubernetes.io/projected/a4951f64-509d-4d21-827c-602b6d8d904d-kube-api-access-m4v49\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.635199 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.635163 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4951f64-509d-4d21-827c-602b6d8d904d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.635199 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.635185 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4951f64-509d-4d21-827c-602b6d8d904d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.635444 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.635413 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4951f64-509d-4d21-827c-602b6d8d904d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.635505 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.635452 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4951f64-509d-4d21-827c-602b6d8d904d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.635560 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.635540 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4951f64-509d-4d21-827c-602b6d8d904d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.637500 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.637476 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4951f64-509d-4d21-827c-602b6d8d904d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.637733 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.637715 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4951f64-509d-4d21-827c-602b6d8d904d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.643248 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.643230 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4v49\" (UniqueName: \"kubernetes.io/projected/a4951f64-509d-4d21-827c-602b6d8d904d-kube-api-access-m4v49\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m9k4g\" (UID: \"a4951f64-509d-4d21-827c-602b6d8d904d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.718370 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.718270 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:02.854979 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:02.854957 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g"] Jul 05 00:24:02.856990 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:24:02.856961 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4951f64_509d_4d21_827c_602b6d8d904d.slice/crio-647171404fae4d66a5d1362ff946f206d204f53af02d8f2cec5d2c061b62913f WatchSource:0}: Error finding container 647171404fae4d66a5d1362ff946f206d204f53af02d8f2cec5d2c061b62913f: Status 404 returned error can't find the container with id 647171404fae4d66a5d1362ff946f206d204f53af02d8f2cec5d2c061b62913f Jul 05 00:24:03.144825 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:03.144786 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" event={"ID":"a4951f64-509d-4d21-827c-602b6d8d904d","Type":"ContainerStarted","Data":"b479380e58bc94490ffedf87c1b9b8c562784ca8247c645079ea755b58ed451a"} Jul 05 00:24:03.145021 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:03.144849 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" event={"ID":"a4951f64-509d-4d21-827c-602b6d8d904d","Type":"ContainerStarted","Data":"647171404fae4d66a5d1362ff946f206d204f53af02d8f2cec5d2c061b62913f"} Jul 05 00:24:03.156456 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:03.156431 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp" Jul 05 00:24:03.162024 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:03.161979 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" podStartSLOduration=0.976773842 podStartE2EDuration="1.161962786s" totalImagesPullingTime="185.188944ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:24:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:24:03.160271785 +0000 UTC m=+881.096637188" watchObservedRunningTime="2026-07-05 00:24:03.161962786 +0000 UTC m=+881.098328167" Jul 05 00:24:12.718408 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:12.718311 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:12.718408 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:12.718371 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:12.732268 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:12.732239 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:13.132031 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:13.131956 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:24:13.202827 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:13.202797 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m9k4g" Jul 05 00:24:20.895172 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:20.895135 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78"] Jul 05 00:24:20.897658 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:20.897642 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:20.900505 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:20.900477 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 05 00:24:20.908531 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:20.908510 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78"] Jul 05 00:24:20.925850 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:20.925824 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:24:20.995822 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:20.995794 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:20.995985 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:20.995866 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:20.995985 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:20.995890 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:20.995985 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:20.995960 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:20.996099 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:20.996005 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:20.996099 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:20.996033 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wl5t\" (UniqueName: \"kubernetes.io/projected/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-kube-api-access-7wl5t\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:21.097224 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:21.097196 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:21.097407 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:21.097241 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:21.097407 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:21.097257 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:21.097407 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:21.097283 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:21.097407 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:21.097331 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:21.097407 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:21.097371 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wl5t\" (UniqueName: \"kubernetes.io/projected/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-kube-api-access-7wl5t\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:21.098163 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:21.098123 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:21.098286 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:21.098142 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:21.098357 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:21.098310 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:21.100573 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:21.100543 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:21.104394 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:21.100940 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:21.106194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:21.106174 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wl5t\" (UniqueName: \"kubernetes.io/projected/bc02abe7-2fe7-4d2f-9158-0ced489b3a00-kube-api-access-7wl5t\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78\" (UID: \"bc02abe7-2fe7-4d2f-9158-0ced489b3a00\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:21.209987 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:21.209883 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:21.343454 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:21.343423 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78"] Jul 05 00:24:21.348966 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:24:21.348760 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc02abe7_2fe7_4d2f_9158_0ced489b3a00.slice/crio-1ed4a940838baf089bfe441001c485d747ea93ba541fd81b8a30b5dbf4e64f26 WatchSource:0}: Error finding container 1ed4a940838baf089bfe441001c485d747ea93ba541fd81b8a30b5dbf4e64f26: Status 404 returned error can't find the container with id 1ed4a940838baf089bfe441001c485d747ea93ba541fd81b8a30b5dbf4e64f26 Jul 05 00:24:22.237813 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:22.237770 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" event={"ID":"bc02abe7-2fe7-4d2f-9158-0ced489b3a00","Type":"ContainerStarted","Data":"834a91b2f866ea8d968485dec7be92a9fdafa24d2fa5634105006d362dcf22d7"} Jul 05 00:24:22.238219 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:22.237838 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" event={"ID":"bc02abe7-2fe7-4d2f-9158-0ced489b3a00","Type":"ContainerStarted","Data":"1ed4a940838baf089bfe441001c485d747ea93ba541fd81b8a30b5dbf4e64f26"} Jul 05 00:24:22.256324 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:22.256278 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" podStartSLOduration=2.097887417 podStartE2EDuration="2.256261024s" totalImagesPullingTime="158.373607ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:24:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:24:22.255161376 +0000 UTC m=+900.191526756" watchObservedRunningTime="2026-07-05 00:24:22.256261024 +0000 UTC m=+900.192626403" Jul 05 00:24:29.482214 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:29.482175 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:24:30.235064 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:30.235028 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:24:31.210735 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:31.210694 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:31.211197 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:31.211008 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:31.223561 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:31.223538 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:31.296076 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:31.296047 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78" Jul 05 00:24:34.827334 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:34.827302 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:24:47.448632 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:24:47.448594 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:25:32.587893 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:25:32.587815 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:25:32.683293 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:25:32.683256 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:25:33.695296 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:25:33.695260 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:25:33.798143 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:25:33.798100 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:26:45.458090 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.458057 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:26:45.559710 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.559669 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-98468dc4b-gcwk5"] Jul 05 00:26:45.559994 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.559966 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-98468dc4b-gcwk5" podUID="061ca742-3d48-4d61-9d27-51ce74c41af3" containerName="manager" containerID="cri-o://619dcb8f97c7ec45d75804cd14673b0e114021c322f1a4df5e76ec3f37879399" gracePeriod=30 Jul 05 00:26:45.813862 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.813838 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:26:45.858295 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.858264 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tqgz\" (UniqueName: \"kubernetes.io/projected/061ca742-3d48-4d61-9d27-51ce74c41af3-kube-api-access-7tqgz\") pod \"061ca742-3d48-4d61-9d27-51ce74c41af3\" (UID: \"061ca742-3d48-4d61-9d27-51ce74c41af3\") " Jul 05 00:26:45.858445 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.858316 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/061ca742-3d48-4d61-9d27-51ce74c41af3-service-ca\") pod \"061ca742-3d48-4d61-9d27-51ce74c41af3\" (UID: \"061ca742-3d48-4d61-9d27-51ce74c41af3\") " Jul 05 00:26:45.858445 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.858380 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/061ca742-3d48-4d61-9d27-51ce74c41af3-webhook-cert\") pod \"061ca742-3d48-4d61-9d27-51ce74c41af3\" (UID: \"061ca742-3d48-4d61-9d27-51ce74c41af3\") " Jul 05 00:26:45.858720 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.858685 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/061ca742-3d48-4d61-9d27-51ce74c41af3-service-ca" (OuterVolumeSpecName: "service-ca") pod "061ca742-3d48-4d61-9d27-51ce74c41af3" (UID: "061ca742-3d48-4d61-9d27-51ce74c41af3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:26:45.860453 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.860433 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/061ca742-3d48-4d61-9d27-51ce74c41af3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "061ca742-3d48-4d61-9d27-51ce74c41af3" (UID: "061ca742-3d48-4d61-9d27-51ce74c41af3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 00:26:45.860579 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.860558 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/061ca742-3d48-4d61-9d27-51ce74c41af3-kube-api-access-7tqgz" (OuterVolumeSpecName: "kube-api-access-7tqgz") pod "061ca742-3d48-4d61-9d27-51ce74c41af3" (UID: "061ca742-3d48-4d61-9d27-51ce74c41af3"). InnerVolumeSpecName "kube-api-access-7tqgz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:26:45.918008 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.917975 2558 generic.go:356] "Generic (PLEG): container finished" podID="061ca742-3d48-4d61-9d27-51ce74c41af3" containerID="619dcb8f97c7ec45d75804cd14673b0e114021c322f1a4df5e76ec3f37879399" exitCode=0 Jul 05 00:26:45.918180 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.918024 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98468dc4b-gcwk5" Jul 05 00:26:45.918180 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.918024 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98468dc4b-gcwk5" event={"ID":"061ca742-3d48-4d61-9d27-51ce74c41af3","Type":"ContainerDied","Data":"619dcb8f97c7ec45d75804cd14673b0e114021c322f1a4df5e76ec3f37879399"} Jul 05 00:26:45.918180 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.918140 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98468dc4b-gcwk5" event={"ID":"061ca742-3d48-4d61-9d27-51ce74c41af3","Type":"ContainerDied","Data":"b861995839e0de3217b248d2f760096f1c10b5507cff166c5e501269146e93c0"} Jul 05 00:26:45.918180 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.918165 2558 scope.go:117] "RemoveContainer" containerID="619dcb8f97c7ec45d75804cd14673b0e114021c322f1a4df5e76ec3f37879399" Jul 05 00:26:45.927408 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.927389 2558 scope.go:117] "RemoveContainer" containerID="619dcb8f97c7ec45d75804cd14673b0e114021c322f1a4df5e76ec3f37879399" Jul 05 00:26:45.927671 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:26:45.927651 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"619dcb8f97c7ec45d75804cd14673b0e114021c322f1a4df5e76ec3f37879399\": container with ID starting with 619dcb8f97c7ec45d75804cd14673b0e114021c322f1a4df5e76ec3f37879399 not found: ID does not exist" containerID="619dcb8f97c7ec45d75804cd14673b0e114021c322f1a4df5e76ec3f37879399" Jul 05 00:26:45.927708 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.927681 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"619dcb8f97c7ec45d75804cd14673b0e114021c322f1a4df5e76ec3f37879399"} err="failed to get container status \"619dcb8f97c7ec45d75804cd14673b0e114021c322f1a4df5e76ec3f37879399\": rpc error: code = NotFound desc = could not find container \"619dcb8f97c7ec45d75804cd14673b0e114021c322f1a4df5e76ec3f37879399\": container with ID starting with 619dcb8f97c7ec45d75804cd14673b0e114021c322f1a4df5e76ec3f37879399 not found: ID does not exist" Jul 05 00:26:45.944499 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.944265 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-98468dc4b-gcwk5"] Jul 05 00:26:45.945579 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.945559 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-98468dc4b-gcwk5"] Jul 05 00:26:45.959121 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.959090 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tqgz\" (UniqueName: \"kubernetes.io/projected/061ca742-3d48-4d61-9d27-51ce74c41af3-kube-api-access-7tqgz\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:26:45.959121 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.959121 2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/061ca742-3d48-4d61-9d27-51ce74c41af3-service-ca\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:26:45.959257 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:45.959135 2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/061ca742-3d48-4d61-9d27-51ce74c41af3-webhook-cert\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:26:46.578154 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.578119 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:26:46.623954 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.623924 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="061ca742-3d48-4d61-9d27-51ce74c41af3" path="/var/lib/kubelet/pods/061ca742-3d48-4d61-9d27-51ce74c41af3/volumes" Jul 05 00:26:46.709128 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.709095 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-98468dc4b-db98h"] Jul 05 00:26:46.709463 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.709451 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="061ca742-3d48-4d61-9d27-51ce74c41af3" containerName="manager" Jul 05 00:26:46.709510 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.709465 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="061ca742-3d48-4d61-9d27-51ce74c41af3" containerName="manager" Jul 05 00:26:46.709543 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.709534 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="061ca742-3d48-4d61-9d27-51ce74c41af3" containerName="manager" Jul 05 00:26:46.713959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.713940 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98468dc4b-db98h" Jul 05 00:26:46.717724 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.717703 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ngppp\"" Jul 05 00:26:46.717845 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.717726 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 05 00:26:46.717845 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.717795 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 05 00:26:46.721960 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.721941 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-98468dc4b-db98h"] Jul 05 00:26:46.767319 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.767287 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae82ae7b-1baf-4c76-b71f-5f3efef3df37-webhook-cert\") pod \"maas-controller-98468dc4b-db98h\" (UID: \"ae82ae7b-1baf-4c76-b71f-5f3efef3df37\") " pod="opendatahub/maas-controller-98468dc4b-db98h" Jul 05 00:26:46.767476 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.767417 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae82ae7b-1baf-4c76-b71f-5f3efef3df37-service-ca\") pod \"maas-controller-98468dc4b-db98h\" (UID: \"ae82ae7b-1baf-4c76-b71f-5f3efef3df37\") " pod="opendatahub/maas-controller-98468dc4b-db98h" Jul 05 00:26:46.767556 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.767532 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx9xt\" (UniqueName: \"kubernetes.io/projected/ae82ae7b-1baf-4c76-b71f-5f3efef3df37-kube-api-access-tx9xt\") pod \"maas-controller-98468dc4b-db98h\" (UID: \"ae82ae7b-1baf-4c76-b71f-5f3efef3df37\") " pod="opendatahub/maas-controller-98468dc4b-db98h" Jul 05 00:26:46.868319 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.868229 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae82ae7b-1baf-4c76-b71f-5f3efef3df37-service-ca\") pod \"maas-controller-98468dc4b-db98h\" (UID: \"ae82ae7b-1baf-4c76-b71f-5f3efef3df37\") " pod="opendatahub/maas-controller-98468dc4b-db98h" Jul 05 00:26:46.868319 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.868292 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx9xt\" (UniqueName: \"kubernetes.io/projected/ae82ae7b-1baf-4c76-b71f-5f3efef3df37-kube-api-access-tx9xt\") pod \"maas-controller-98468dc4b-db98h\" (UID: \"ae82ae7b-1baf-4c76-b71f-5f3efef3df37\") " pod="opendatahub/maas-controller-98468dc4b-db98h" Jul 05 00:26:46.868519 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.868329 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae82ae7b-1baf-4c76-b71f-5f3efef3df37-webhook-cert\") pod \"maas-controller-98468dc4b-db98h\" (UID: \"ae82ae7b-1baf-4c76-b71f-5f3efef3df37\") " pod="opendatahub/maas-controller-98468dc4b-db98h" Jul 05 00:26:46.868862 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.868839 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae82ae7b-1baf-4c76-b71f-5f3efef3df37-service-ca\") pod \"maas-controller-98468dc4b-db98h\" (UID: \"ae82ae7b-1baf-4c76-b71f-5f3efef3df37\") " pod="opendatahub/maas-controller-98468dc4b-db98h" Jul 05 00:26:46.870864 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.870841 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae82ae7b-1baf-4c76-b71f-5f3efef3df37-webhook-cert\") pod \"maas-controller-98468dc4b-db98h\" (UID: \"ae82ae7b-1baf-4c76-b71f-5f3efef3df37\") " pod="opendatahub/maas-controller-98468dc4b-db98h" Jul 05 00:26:46.876065 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:46.876038 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx9xt\" (UniqueName: \"kubernetes.io/projected/ae82ae7b-1baf-4c76-b71f-5f3efef3df37-kube-api-access-tx9xt\") pod \"maas-controller-98468dc4b-db98h\" (UID: \"ae82ae7b-1baf-4c76-b71f-5f3efef3df37\") " pod="opendatahub/maas-controller-98468dc4b-db98h" Jul 05 00:26:47.025133 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:47.025104 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98468dc4b-db98h" Jul 05 00:26:47.157121 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:47.157098 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-98468dc4b-db98h"] Jul 05 00:26:47.159330 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:26:47.159306 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae82ae7b_1baf_4c76_b71f_5f3efef3df37.slice/crio-b0b3de693b9624c691f19bb5d1576042465441ac859b73589e3a79b3d500256a WatchSource:0}: Error finding container b0b3de693b9624c691f19bb5d1576042465441ac859b73589e3a79b3d500256a: Status 404 returned error can't find the container with id b0b3de693b9624c691f19bb5d1576042465441ac859b73589e3a79b3d500256a Jul 05 00:26:47.160748 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:47.160725 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 00:26:47.930411 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:47.930378 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98468dc4b-db98h" event={"ID":"ae82ae7b-1baf-4c76-b71f-5f3efef3df37","Type":"ContainerStarted","Data":"457db27581fb79cbcbeeea683361ace6fdab1fc56c16c1721ef5e222becb0146"} Jul 05 00:26:47.930799 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:47.930431 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98468dc4b-db98h" event={"ID":"ae82ae7b-1baf-4c76-b71f-5f3efef3df37","Type":"ContainerStarted","Data":"b0b3de693b9624c691f19bb5d1576042465441ac859b73589e3a79b3d500256a"} Jul 05 00:26:47.930799 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:47.930518 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-98468dc4b-db98h" Jul 05 00:26:47.946792 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:47.946750 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-98468dc4b-db98h" podStartSLOduration=1.399478395 podStartE2EDuration="1.946736926s" totalImagesPullingTime="547.258531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:26:47.94526803 +0000 UTC m=+1045.881633422" watchObservedRunningTime="2026-07-05 00:26:47.946736926 +0000 UTC m=+1045.883102304" Jul 05 00:26:58.939963 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:26:58.939930 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-98468dc4b-db98h" Jul 05 00:27:02.234502 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:27:02.234465 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:27:22.051948 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:27:22.051836 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:27:22.152409 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:27:22.152375 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:27:22.271335 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:27:22.271306 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:27:27.870007 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:27:27.869970 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:27:27.975679 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:27:27.975637 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:27:28.070416 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:27:28.070384 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:27:28.221717 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:27:28.221680 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:27:33.784986 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:27:33.784955 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:27:33.844103 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:27:33.844068 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:28:28.164818 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:28:28.164786 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:28:58.892963 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:28:58.892864 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:30:10.382179 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:30:10.382135 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:31:15.076644 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:31:15.076613 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:35:15.167035 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:35:15.167002 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:35:45.451403 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:35:45.451368 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:37:25.334765 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:37:25.334737 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:38:31.746419 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:38:31.746367 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:38:41.294151 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:38:41.294115 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:38:49.552047 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:38:49.552011 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:38:54.658275 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:38:54.658243 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:38:57.417367 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:38:57.417320 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:38:57.520436 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:38:57.520402 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:38:57.865668 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:38:57.865588 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:03.505790 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:03.505753 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:03.565118 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:03.565081 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:03.690953 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:03.690889 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:03.744662 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:03.744629 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:04.889494 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:04.889431 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:04.939084 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:04.939053 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf"] Jul 05 00:39:04.943144 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:04.943124 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:04.945725 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:04.945703 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-84731c49-serving-cert\"" Jul 05 00:39:04.950008 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:04.949988 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf"] Jul 05 00:39:04.965696 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:04.965668 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91cf2eff-72ca-4211-b15a-c19ffedde7c1-maas-api-tls\") pod \"maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf\" (UID: \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\") " pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:04.965696 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:04.965702 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91cf2eff-72ca-4211-b15a-c19ffedde7c1-openshift-service-ca\") pod \"maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf\" (UID: \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\") " pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:04.965936 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:04.965815 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dctp\" (UniqueName: \"kubernetes.io/projected/91cf2eff-72ca-4211-b15a-c19ffedde7c1-kube-api-access-6dctp\") pod \"maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf\" (UID: \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\") " pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:04.966831 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:04.966767 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:05.031007 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:05.030973 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:05.066305 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:05.066280 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dctp\" (UniqueName: \"kubernetes.io/projected/91cf2eff-72ca-4211-b15a-c19ffedde7c1-kube-api-access-6dctp\") pod \"maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf\" (UID: \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\") " pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:05.066441 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:05.066326 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91cf2eff-72ca-4211-b15a-c19ffedde7c1-maas-api-tls\") pod \"maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf\" (UID: \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\") " pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:05.066441 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:05.066343 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91cf2eff-72ca-4211-b15a-c19ffedde7c1-openshift-service-ca\") pod \"maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf\" (UID: \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\") " pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:05.066531 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:39:05.066435 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-84731c49-serving-cert: secret "maas-api-e2e-ait-84731c49-serving-cert" not found Jul 05 00:39:05.066531 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:39:05.066501 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91cf2eff-72ca-4211-b15a-c19ffedde7c1-maas-api-tls podName:91cf2eff-72ca-4211-b15a-c19ffedde7c1 nodeName:}" failed. No retries permitted until 2026-07-05 00:39:05.566484195 +0000 UTC m=+1783.502849551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/91cf2eff-72ca-4211-b15a-c19ffedde7c1-maas-api-tls") pod "maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" (UID: "91cf2eff-72ca-4211-b15a-c19ffedde7c1") : secret "maas-api-e2e-ait-84731c49-serving-cert" not found Jul 05 00:39:05.067050 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:05.067030 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91cf2eff-72ca-4211-b15a-c19ffedde7c1-openshift-service-ca\") pod \"maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf\" (UID: \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\") " pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:05.076226 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:05.076203 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dctp\" (UniqueName: \"kubernetes.io/projected/91cf2eff-72ca-4211-b15a-c19ffedde7c1-kube-api-access-6dctp\") pod \"maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf\" (UID: \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\") " pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:05.142559 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:05.142488 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:05.412699 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:05.412611 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:05.512136 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:05.512104 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:05.570557 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:05.570528 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91cf2eff-72ca-4211-b15a-c19ffedde7c1-maas-api-tls\") pod \"maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf\" (UID: \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\") " pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:05.573125 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:05.573107 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91cf2eff-72ca-4211-b15a-c19ffedde7c1-maas-api-tls\") pod \"maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf\" (UID: \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\") " pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:05.860537 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:05.860460 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:05.994525 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:05.994501 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf"] Jul 05 00:39:05.996628 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:39:05.996602 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91cf2eff_72ca_4211_b15a_c19ffedde7c1.slice/crio-3a0854164c0cbcdffdcc5c6460c756faa483f1bf1f85e6ffbbe579f90f674e24 WatchSource:0}: Error finding container 3a0854164c0cbcdffdcc5c6460c756faa483f1bf1f85e6ffbbe579f90f674e24: Status 404 returned error can't find the container with id 3a0854164c0cbcdffdcc5c6460c756faa483f1bf1f85e6ffbbe579f90f674e24 Jul 05 00:39:05.997950 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:05.997932 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 00:39:06.420481 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:06.420448 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" event={"ID":"91cf2eff-72ca-4211-b15a-c19ffedde7c1","Type":"ContainerStarted","Data":"3a0854164c0cbcdffdcc5c6460c756faa483f1bf1f85e6ffbbe579f90f674e24"} Jul 05 00:39:06.993326 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:06.993266 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:07.427020 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:07.426983 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" event={"ID":"91cf2eff-72ca-4211-b15a-c19ffedde7c1","Type":"ContainerStarted","Data":"5c91ae28108360a85bbaf04f601f9a91996ac351b0c67189d749f1c3a2c5227d"} Jul 05 00:39:07.427469 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:07.427127 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:07.444162 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:07.444115 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" podStartSLOduration=2.9089473999999997 podStartE2EDuration="3.444101745s" totalImagesPullingTime="535.154345ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:39:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:39:07.44237478 +0000 UTC m=+1785.378740157" watchObservedRunningTime="2026-07-05 00:39:07.444101745 +0000 UTC m=+1785.380467122" Jul 05 00:39:12.965602 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:12.965509 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf"] Jul 05 00:39:12.966026 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:12.965737 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" podUID="91cf2eff-72ca-4211-b15a-c19ffedde7c1" containerName="maas-api" containerID="cri-o://5c91ae28108360a85bbaf04f601f9a91996ac351b0c67189d749f1c3a2c5227d" gracePeriod=30 Jul 05 00:39:12.972260 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:12.972236 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:12.977371 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:12.977333 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:13.044226 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.044191 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:13.141698 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.141449 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:13.217777 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.217713 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:13.237318 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.237282 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:13.240276 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.240254 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dctp\" (UniqueName: \"kubernetes.io/projected/91cf2eff-72ca-4211-b15a-c19ffedde7c1-kube-api-access-6dctp\") pod \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\" (UID: \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\") " Jul 05 00:39:13.240404 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.240346 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91cf2eff-72ca-4211-b15a-c19ffedde7c1-openshift-service-ca\") pod \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\" (UID: \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\") " Jul 05 00:39:13.240404 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.240373 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91cf2eff-72ca-4211-b15a-c19ffedde7c1-maas-api-tls\") pod \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\" (UID: \"91cf2eff-72ca-4211-b15a-c19ffedde7c1\") " Jul 05 00:39:13.240765 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.240736 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91cf2eff-72ca-4211-b15a-c19ffedde7c1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "91cf2eff-72ca-4211-b15a-c19ffedde7c1" (UID: "91cf2eff-72ca-4211-b15a-c19ffedde7c1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:39:13.243124 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.243095 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91cf2eff-72ca-4211-b15a-c19ffedde7c1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "91cf2eff-72ca-4211-b15a-c19ffedde7c1" (UID: "91cf2eff-72ca-4211-b15a-c19ffedde7c1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 00:39:13.243216 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.243197 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91cf2eff-72ca-4211-b15a-c19ffedde7c1-kube-api-access-6dctp" (OuterVolumeSpecName: "kube-api-access-6dctp") pod "91cf2eff-72ca-4211-b15a-c19ffedde7c1" (UID: "91cf2eff-72ca-4211-b15a-c19ffedde7c1"). InnerVolumeSpecName "kube-api-access-6dctp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:39:13.341393 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.341364 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91cf2eff-72ca-4211-b15a-c19ffedde7c1-openshift-service-ca\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:39:13.341393 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.341391 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91cf2eff-72ca-4211-b15a-c19ffedde7c1-maas-api-tls\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:39:13.341393 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.341400 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dctp\" (UniqueName: \"kubernetes.io/projected/91cf2eff-72ca-4211-b15a-c19ffedde7c1-kube-api-access-6dctp\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:39:13.455310 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.455269 2558 generic.go:356] "Generic (PLEG): container finished" podID="91cf2eff-72ca-4211-b15a-c19ffedde7c1" containerID="5c91ae28108360a85bbaf04f601f9a91996ac351b0c67189d749f1c3a2c5227d" exitCode=0 Jul 05 00:39:13.455310 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.455316 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" Jul 05 00:39:13.455548 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.455351 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" event={"ID":"91cf2eff-72ca-4211-b15a-c19ffedde7c1","Type":"ContainerDied","Data":"5c91ae28108360a85bbaf04f601f9a91996ac351b0c67189d749f1c3a2c5227d"} Jul 05 00:39:13.455548 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.455412 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf" event={"ID":"91cf2eff-72ca-4211-b15a-c19ffedde7c1","Type":"ContainerDied","Data":"3a0854164c0cbcdffdcc5c6460c756faa483f1bf1f85e6ffbbe579f90f674e24"} Jul 05 00:39:13.455548 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.455444 2558 scope.go:117] "RemoveContainer" containerID="5c91ae28108360a85bbaf04f601f9a91996ac351b0c67189d749f1c3a2c5227d" Jul 05 00:39:13.464103 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.464086 2558 scope.go:117] "RemoveContainer" containerID="5c91ae28108360a85bbaf04f601f9a91996ac351b0c67189d749f1c3a2c5227d" Jul 05 00:39:13.464369 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:39:13.464350 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c91ae28108360a85bbaf04f601f9a91996ac351b0c67189d749f1c3a2c5227d\": container with ID starting with 5c91ae28108360a85bbaf04f601f9a91996ac351b0c67189d749f1c3a2c5227d not found: ID does not exist" containerID="5c91ae28108360a85bbaf04f601f9a91996ac351b0c67189d749f1c3a2c5227d" Jul 05 00:39:13.464422 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.464376 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c91ae28108360a85bbaf04f601f9a91996ac351b0c67189d749f1c3a2c5227d"} err="failed to get container status \"5c91ae28108360a85bbaf04f601f9a91996ac351b0c67189d749f1c3a2c5227d\": rpc error: code = NotFound desc = could not find container \"5c91ae28108360a85bbaf04f601f9a91996ac351b0c67189d749f1c3a2c5227d\": container with ID starting with 5c91ae28108360a85bbaf04f601f9a91996ac351b0c67189d749f1c3a2c5227d not found: ID does not exist" Jul 05 00:39:13.476195 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.476132 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf"] Jul 05 00:39:13.478109 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:13.478090 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-84731c49-7c99b7d5f4-lt8zf"] Jul 05 00:39:14.624987 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:14.624959 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91cf2eff-72ca-4211-b15a-c19ffedde7c1" path="/var/lib/kubelet/pods/91cf2eff-72ca-4211-b15a-c19ffedde7c1/volumes" Jul 05 00:39:14.630360 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:14.630313 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:18.866288 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:18.866256 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:18.967157 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:18.967121 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:19.079543 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:19.079508 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:19.180301 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:19.180262 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:20.254483 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.254437 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:20.304178 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.304138 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65"] Jul 05 00:39:20.304720 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.304703 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91cf2eff-72ca-4211-b15a-c19ffedde7c1" containerName="maas-api" Jul 05 00:39:20.304720 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.304722 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="91cf2eff-72ca-4211-b15a-c19ffedde7c1" containerName="maas-api" Jul 05 00:39:20.304864 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.304806 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="91cf2eff-72ca-4211-b15a-c19ffedde7c1" containerName="maas-api" Jul 05 00:39:20.308239 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.308216 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:20.311276 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.311253 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-8c1a3081-serving-cert\"" Jul 05 00:39:20.318654 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.318631 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65"] Jul 05 00:39:20.335633 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.335594 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:20.396627 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.396598 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:20.398698 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.398678 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121c29db-4f2e-4e03-ae0e-a928eeee1b36-openshift-service-ca\") pod \"maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65\" (UID: \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\") " pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:20.398792 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.398756 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121c29db-4f2e-4e03-ae0e-a928eeee1b36-maas-api-tls\") pod \"maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65\" (UID: \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\") " pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:20.398872 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.398855 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccwpw\" (UniqueName: \"kubernetes.io/projected/121c29db-4f2e-4e03-ae0e-a928eeee1b36-kube-api-access-ccwpw\") pod \"maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65\" (UID: \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\") " pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:20.500217 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.500186 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121c29db-4f2e-4e03-ae0e-a928eeee1b36-maas-api-tls\") pod \"maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65\" (UID: \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\") " pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:20.500436 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.500289 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccwpw\" (UniqueName: \"kubernetes.io/projected/121c29db-4f2e-4e03-ae0e-a928eeee1b36-kube-api-access-ccwpw\") pod \"maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65\" (UID: \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\") " pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:20.500436 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.500339 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121c29db-4f2e-4e03-ae0e-a928eeee1b36-openshift-service-ca\") pod \"maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65\" (UID: \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\") " pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:20.501012 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.500943 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121c29db-4f2e-4e03-ae0e-a928eeee1b36-openshift-service-ca\") pod \"maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65\" (UID: \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\") " pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:20.502813 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.502784 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121c29db-4f2e-4e03-ae0e-a928eeee1b36-maas-api-tls\") pod \"maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65\" (UID: \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\") " pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:20.507996 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.507941 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccwpw\" (UniqueName: \"kubernetes.io/projected/121c29db-4f2e-4e03-ae0e-a928eeee1b36-kube-api-access-ccwpw\") pod \"maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65\" (UID: \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\") " pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:20.509114 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.509093 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:20.621785 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.621744 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:20.763533 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.763454 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65"] Jul 05 00:39:20.766719 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:39:20.766694 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod121c29db_4f2e_4e03_ae0e_a928eeee1b36.slice/crio-7de80193daf35d5dcb1809dff9b425ab7a5a720645838805c13dd99719c4d68b WatchSource:0}: Error finding container 7de80193daf35d5dcb1809dff9b425ab7a5a720645838805c13dd99719c4d68b: Status 404 returned error can't find the container with id 7de80193daf35d5dcb1809dff9b425ab7a5a720645838805c13dd99719c4d68b Jul 05 00:39:20.893380 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.893349 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:20.994630 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:20.994600 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:21.495643 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:21.495610 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" event={"ID":"121c29db-4f2e-4e03-ae0e-a928eeee1b36","Type":"ContainerStarted","Data":"941283f5ea7e9a00513e1d2b22abfd3e62b918bff4fa7534a32df6dc81c9564d"} Jul 05 00:39:21.496086 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:21.495659 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" event={"ID":"121c29db-4f2e-4e03-ae0e-a928eeee1b36","Type":"ContainerStarted","Data":"7de80193daf35d5dcb1809dff9b425ab7a5a720645838805c13dd99719c4d68b"} Jul 05 00:39:21.496086 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:21.495690 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:21.512299 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:21.512254 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" podStartSLOduration=1.14336525 podStartE2EDuration="1.512240365s" totalImagesPullingTime="368.875115ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:39:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:39:21.511333281 +0000 UTC m=+1799.447698676" watchObservedRunningTime="2026-07-05 00:39:21.512240365 +0000 UTC m=+1799.448605743" Jul 05 00:39:22.063867 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:22.063831 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:27.505646 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:27.505616 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:28.363563 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.363532 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65"] Jul 05 00:39:28.363759 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.363731 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" podUID="121c29db-4f2e-4e03-ae0e-a928eeee1b36" containerName="maas-api" containerID="cri-o://941283f5ea7e9a00513e1d2b22abfd3e62b918bff4fa7534a32df6dc81c9564d" gracePeriod=30 Jul 05 00:39:28.386325 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.386291 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:28.487689 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.487659 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:28.531278 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.531238 2558 generic.go:356] "Generic (PLEG): container finished" podID="121c29db-4f2e-4e03-ae0e-a928eeee1b36" containerID="941283f5ea7e9a00513e1d2b22abfd3e62b918bff4fa7534a32df6dc81c9564d" exitCode=0 Jul 05 00:39:28.531651 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.531321 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" event={"ID":"121c29db-4f2e-4e03-ae0e-a928eeee1b36","Type":"ContainerDied","Data":"941283f5ea7e9a00513e1d2b22abfd3e62b918bff4fa7534a32df6dc81c9564d"} Jul 05 00:39:28.616435 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.616379 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:28.672753 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.672718 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121c29db-4f2e-4e03-ae0e-a928eeee1b36-maas-api-tls\") pod \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\" (UID: \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\") " Jul 05 00:39:28.672753 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.672756 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccwpw\" (UniqueName: \"kubernetes.io/projected/121c29db-4f2e-4e03-ae0e-a928eeee1b36-kube-api-access-ccwpw\") pod \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\" (UID: \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\") " Jul 05 00:39:28.673033 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.672783 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121c29db-4f2e-4e03-ae0e-a928eeee1b36-openshift-service-ca\") pod \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\" (UID: \"121c29db-4f2e-4e03-ae0e-a928eeee1b36\") " Jul 05 00:39:28.673178 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.673156 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/121c29db-4f2e-4e03-ae0e-a928eeee1b36-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "121c29db-4f2e-4e03-ae0e-a928eeee1b36" (UID: "121c29db-4f2e-4e03-ae0e-a928eeee1b36"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:39:28.675069 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.675042 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/121c29db-4f2e-4e03-ae0e-a928eeee1b36-kube-api-access-ccwpw" (OuterVolumeSpecName: "kube-api-access-ccwpw") pod "121c29db-4f2e-4e03-ae0e-a928eeee1b36" (UID: "121c29db-4f2e-4e03-ae0e-a928eeee1b36"). InnerVolumeSpecName "kube-api-access-ccwpw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:39:28.675174 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.675146 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/121c29db-4f2e-4e03-ae0e-a928eeee1b36-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "121c29db-4f2e-4e03-ae0e-a928eeee1b36" (UID: "121c29db-4f2e-4e03-ae0e-a928eeee1b36"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 00:39:28.774279 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.774246 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121c29db-4f2e-4e03-ae0e-a928eeee1b36-maas-api-tls\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:39:28.774279 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.774272 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccwpw\" (UniqueName: \"kubernetes.io/projected/121c29db-4f2e-4e03-ae0e-a928eeee1b36-kube-api-access-ccwpw\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:39:28.774279 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:28.774281 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121c29db-4f2e-4e03-ae0e-a928eeee1b36-openshift-service-ca\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:39:29.537311 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:29.537282 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" Jul 05 00:39:29.537742 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:29.537325 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65" event={"ID":"121c29db-4f2e-4e03-ae0e-a928eeee1b36","Type":"ContainerDied","Data":"7de80193daf35d5dcb1809dff9b425ab7a5a720645838805c13dd99719c4d68b"} Jul 05 00:39:29.537742 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:29.537385 2558 scope.go:117] "RemoveContainer" containerID="941283f5ea7e9a00513e1d2b22abfd3e62b918bff4fa7534a32df6dc81c9564d" Jul 05 00:39:29.556576 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:29.556523 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:29.561591 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:29.561566 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65"] Jul 05 00:39:29.563345 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:29.563326 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-8c1a3081-d9745c68d-rgq65"] Jul 05 00:39:29.657348 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:29.657312 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:30.623853 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:30.623822 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="121c29db-4f2e-4e03-ae0e-a928eeee1b36" path="/var/lib/kubelet/pods/121c29db-4f2e-4e03-ae0e-a928eeee1b36/volumes" Jul 05 00:39:33.398330 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:33.398300 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:35.369723 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:35.369683 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:35.420787 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:35.420748 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:35.659265 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:35.659234 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:35.731111 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:35.731076 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:36.773645 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:36.773601 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:36.845229 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:36.845196 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz"] Jul 05 00:39:36.845765 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:36.845743 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="121c29db-4f2e-4e03-ae0e-a928eeee1b36" containerName="maas-api" Jul 05 00:39:36.845765 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:36.845764 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="121c29db-4f2e-4e03-ae0e-a928eeee1b36" containerName="maas-api" Jul 05 00:39:36.846001 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:36.845875 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="121c29db-4f2e-4e03-ae0e-a928eeee1b36" containerName="maas-api" Jul 05 00:39:36.850518 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:36.850494 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:36.853734 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:36.853130 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-e1765800-serving-cert\"" Jul 05 00:39:36.856468 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:36.856411 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz"] Jul 05 00:39:36.869383 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:36.869360 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:36.927346 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:36.927316 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:36.953079 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:36.953046 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-openshift-service-ca\") pod \"maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz\" (UID: \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\") " pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:36.953220 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:36.953113 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-maas-api-tls\") pod \"maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz\" (UID: \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\") " pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:36.953220 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:36.953169 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sq9f\" (UniqueName: \"kubernetes.io/projected/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-kube-api-access-2sq9f\") pod \"maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz\" (UID: \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\") " pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:37.054630 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:37.054555 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-maas-api-tls\") pod \"maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz\" (UID: \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\") " pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:37.054630 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:37.054588 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sq9f\" (UniqueName: \"kubernetes.io/projected/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-kube-api-access-2sq9f\") pod \"maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz\" (UID: \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\") " pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:37.054834 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:37.054792 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-openshift-service-ca\") pod \"maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz\" (UID: \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\") " pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:37.055407 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:37.055383 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-openshift-service-ca\") pod \"maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz\" (UID: \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\") " pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:37.057153 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:37.057135 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-maas-api-tls\") pod \"maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz\" (UID: \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\") " pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:37.064559 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:37.064540 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sq9f\" (UniqueName: \"kubernetes.io/projected/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-kube-api-access-2sq9f\") pod \"maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz\" (UID: \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\") " pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:37.163453 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:37.163417 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:37.297404 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:37.297381 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz"] Jul 05 00:39:37.299823 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:39:37.299794 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf34bbd2b_cf0d_46ef_a4dc_452e4e0a1c6c.slice/crio-19894b0e080e0d0067d56de157d967795f65e60ebe3224ff64e0e3187d91b5c7 WatchSource:0}: Error finding container 19894b0e080e0d0067d56de157d967795f65e60ebe3224ff64e0e3187d91b5c7: Status 404 returned error can't find the container with id 19894b0e080e0d0067d56de157d967795f65e60ebe3224ff64e0e3187d91b5c7 Jul 05 00:39:37.399495 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:37.399464 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:37.500314 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:37.500283 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:37.577366 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:37.577336 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" event={"ID":"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c","Type":"ContainerStarted","Data":"19894b0e080e0d0067d56de157d967795f65e60ebe3224ff64e0e3187d91b5c7"} Jul 05 00:39:38.583838 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:38.583793 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" event={"ID":"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c","Type":"ContainerStarted","Data":"ce753daa81a7b4d53d164f9ea6e0d920d58305c9a2c3fdf82355361ea57f8cb6"} Jul 05 00:39:38.584256 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:38.583920 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:38.601068 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:38.601024 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" podStartSLOduration=2.197949689 podStartE2EDuration="2.601009196s" totalImagesPullingTime="403.059507ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:39:38.600134639 +0000 UTC m=+1816.536500016" watchObservedRunningTime="2026-07-05 00:39:38.601009196 +0000 UTC m=+1816.537374573" Jul 05 00:39:39.041517 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:39.041484 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:44.198690 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.198654 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz"] Jul 05 00:39:44.199097 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.198878 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" podUID="f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c" containerName="maas-api" containerID="cri-o://ce753daa81a7b4d53d164f9ea6e0d920d58305c9a2c3fdf82355361ea57f8cb6" gracePeriod=30 Jul 05 00:39:44.205107 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.205082 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:44.211373 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.211344 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:44.312559 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.312499 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:44.383059 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.383027 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:44.454842 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.454791 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:44.480695 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.480651 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:44.612987 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.612953 2558 generic.go:356] "Generic (PLEG): container finished" podID="f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c" containerID="ce753daa81a7b4d53d164f9ea6e0d920d58305c9a2c3fdf82355361ea57f8cb6" exitCode=0 Jul 05 00:39:44.613140 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.613002 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" Jul 05 00:39:44.613140 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.613040 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" event={"ID":"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c","Type":"ContainerDied","Data":"ce753daa81a7b4d53d164f9ea6e0d920d58305c9a2c3fdf82355361ea57f8cb6"} Jul 05 00:39:44.613140 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.613113 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz" event={"ID":"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c","Type":"ContainerDied","Data":"19894b0e080e0d0067d56de157d967795f65e60ebe3224ff64e0e3187d91b5c7"} Jul 05 00:39:44.613140 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.613136 2558 scope.go:117] "RemoveContainer" containerID="ce753daa81a7b4d53d164f9ea6e0d920d58305c9a2c3fdf82355361ea57f8cb6" Jul 05 00:39:44.620781 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.620743 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-openshift-service-ca\") pod \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\" (UID: \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\") " Jul 05 00:39:44.620781 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.620782 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sq9f\" (UniqueName: \"kubernetes.io/projected/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-kube-api-access-2sq9f\") pod \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\" (UID: \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\") " Jul 05 00:39:44.621010 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.620809 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-maas-api-tls\") pod \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\" (UID: \"f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c\") " Jul 05 00:39:44.621133 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.621109 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c" (UID: "f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:39:44.621304 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.621284 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-openshift-service-ca\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:39:44.621679 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.621657 2558 scope.go:117] "RemoveContainer" containerID="ce753daa81a7b4d53d164f9ea6e0d920d58305c9a2c3fdf82355361ea57f8cb6" Jul 05 00:39:44.622047 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:39:44.622025 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce753daa81a7b4d53d164f9ea6e0d920d58305c9a2c3fdf82355361ea57f8cb6\": container with ID starting with ce753daa81a7b4d53d164f9ea6e0d920d58305c9a2c3fdf82355361ea57f8cb6 not found: ID does not exist" containerID="ce753daa81a7b4d53d164f9ea6e0d920d58305c9a2c3fdf82355361ea57f8cb6" Jul 05 00:39:44.622136 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.622057 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce753daa81a7b4d53d164f9ea6e0d920d58305c9a2c3fdf82355361ea57f8cb6"} err="failed to get container status \"ce753daa81a7b4d53d164f9ea6e0d920d58305c9a2c3fdf82355361ea57f8cb6\": rpc error: code = NotFound desc = could not find container \"ce753daa81a7b4d53d164f9ea6e0d920d58305c9a2c3fdf82355361ea57f8cb6\": container with ID starting with ce753daa81a7b4d53d164f9ea6e0d920d58305c9a2c3fdf82355361ea57f8cb6 not found: ID does not exist" Jul 05 00:39:44.622936 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.622892 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c" (UID: "f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 00:39:44.623117 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.623100 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-kube-api-access-2sq9f" (OuterVolumeSpecName: "kube-api-access-2sq9f") pod "f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c" (UID: "f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c"). InnerVolumeSpecName "kube-api-access-2sq9f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:39:44.722095 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.722029 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sq9f\" (UniqueName: \"kubernetes.io/projected/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-kube-api-access-2sq9f\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:39:44.722095 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.722054 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c-maas-api-tls\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:39:44.927862 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.927820 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz"] Jul 05 00:39:44.931198 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:44.931174 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-e1765800-6b4fdb9454-thkdz"] Jul 05 00:39:46.379073 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:46.379036 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:46.624505 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:46.624430 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c" path="/var/lib/kubelet/pods/f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c/volumes" Jul 05 00:39:50.131961 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:50.131922 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:50.192596 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:50.192563 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:50.614460 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:50.614420 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:50.725588 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:50.725544 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:50.815598 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:50.815566 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:52.155287 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:52.155256 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:57.830049 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:57.830013 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:57.888454 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:57.888419 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:57.956698 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:57.956661 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:58.062218 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:58.062184 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:39:58.503283 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:39:58.503242 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:40:07.153051 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:40:07.152998 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:40:24.449352 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:40:24.449317 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:40:30.025039 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:40:30.025002 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:40:30.126782 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:40:30.126742 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:40:30.222564 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:40:30.222522 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:40:46.112733 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:40:46.112644 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:40:51.864051 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:40:51.864018 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:40:51.922143 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:40:51.922112 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:40:51.991722 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:40:51.991682 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:40:52.092386 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:40:52.092345 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:02.612362 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:02.612324 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:03.381092 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:03.381057 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:09.019256 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:09.019216 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:09.076177 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:09.076145 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:09.165613 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:09.165572 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:09.254599 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:09.254568 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:09.355514 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:09.355424 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:10.074649 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:10.074618 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:10.167827 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:10.167775 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:10.268431 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:10.268397 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:11.803365 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:11.803331 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:17.415055 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:17.415003 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:17.515813 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:17.515778 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:18.083723 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:18.083686 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:23.678522 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:23.678465 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:23.737211 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:23.737179 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:23.917400 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:23.917365 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:24.220264 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:24.220230 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:24.388296 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:24.388258 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:27.297397 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.297344 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:27.358452 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.358425 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd"] Jul 05 00:41:27.358823 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.358810 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c" containerName="maas-api" Jul 05 00:41:27.358870 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.358826 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c" containerName="maas-api" Jul 05 00:41:27.358929 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.358921 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f34bbd2b-cf0d-46ef-a4dc-452e4e0a1c6c" containerName="maas-api" Jul 05 00:41:27.361929 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.361913 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:27.364518 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.364496 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-125bebca-serving-cert\"" Jul 05 00:41:27.373918 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.373882 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd"] Jul 05 00:41:27.375827 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.375786 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:27.434211 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.434175 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:27.496592 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.496557 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgmfp\" (UniqueName: \"kubernetes.io/projected/d3654b0a-f6d2-407f-b117-a371c380df55-kube-api-access-hgmfp\") pod \"maas-api-e2e-mt-125bebca-57c6bf6955-78dxd\" (UID: \"d3654b0a-f6d2-407f-b117-a371c380df55\") " pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:27.496737 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.496634 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3654b0a-f6d2-407f-b117-a371c380df55-maas-api-tls\") pod \"maas-api-e2e-mt-125bebca-57c6bf6955-78dxd\" (UID: \"d3654b0a-f6d2-407f-b117-a371c380df55\") " pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:27.496737 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.496655 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3654b0a-f6d2-407f-b117-a371c380df55-openshift-service-ca\") pod \"maas-api-e2e-mt-125bebca-57c6bf6955-78dxd\" (UID: \"d3654b0a-f6d2-407f-b117-a371c380df55\") " pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:27.597396 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.597320 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgmfp\" (UniqueName: \"kubernetes.io/projected/d3654b0a-f6d2-407f-b117-a371c380df55-kube-api-access-hgmfp\") pod \"maas-api-e2e-mt-125bebca-57c6bf6955-78dxd\" (UID: \"d3654b0a-f6d2-407f-b117-a371c380df55\") " pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:27.597396 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.597384 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3654b0a-f6d2-407f-b117-a371c380df55-maas-api-tls\") pod \"maas-api-e2e-mt-125bebca-57c6bf6955-78dxd\" (UID: \"d3654b0a-f6d2-407f-b117-a371c380df55\") " pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:27.597574 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.597403 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3654b0a-f6d2-407f-b117-a371c380df55-openshift-service-ca\") pod \"maas-api-e2e-mt-125bebca-57c6bf6955-78dxd\" (UID: \"d3654b0a-f6d2-407f-b117-a371c380df55\") " pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:27.597574 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:41:27.597517 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-125bebca-serving-cert: secret "maas-api-e2e-mt-125bebca-serving-cert" not found Jul 05 00:41:27.597653 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:41:27.597589 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3654b0a-f6d2-407f-b117-a371c380df55-maas-api-tls podName:d3654b0a-f6d2-407f-b117-a371c380df55 nodeName:}" failed. No retries permitted until 2026-07-05 00:41:28.097570138 +0000 UTC m=+1926.033935496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d3654b0a-f6d2-407f-b117-a371c380df55-maas-api-tls") pod "maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" (UID: "d3654b0a-f6d2-407f-b117-a371c380df55") : secret "maas-api-e2e-mt-125bebca-serving-cert" not found Jul 05 00:41:27.597970 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.597954 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3654b0a-f6d2-407f-b117-a371c380df55-openshift-service-ca\") pod \"maas-api-e2e-mt-125bebca-57c6bf6955-78dxd\" (UID: \"d3654b0a-f6d2-407f-b117-a371c380df55\") " pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:27.611316 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.611288 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgmfp\" (UniqueName: \"kubernetes.io/projected/d3654b0a-f6d2-407f-b117-a371c380df55-kube-api-access-hgmfp\") pod \"maas-api-e2e-mt-125bebca-57c6bf6955-78dxd\" (UID: \"d3654b0a-f6d2-407f-b117-a371c380df55\") " pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:27.719964 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.719928 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:27.835231 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.835201 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:27.936167 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:27.936139 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:28.101116 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:28.101091 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3654b0a-f6d2-407f-b117-a371c380df55-maas-api-tls\") pod \"maas-api-e2e-mt-125bebca-57c6bf6955-78dxd\" (UID: \"d3654b0a-f6d2-407f-b117-a371c380df55\") " pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:28.103631 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:28.103609 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3654b0a-f6d2-407f-b117-a371c380df55-maas-api-tls\") pod \"maas-api-e2e-mt-125bebca-57c6bf6955-78dxd\" (UID: \"d3654b0a-f6d2-407f-b117-a371c380df55\") " pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:28.275065 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:28.274982 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:28.404471 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:28.404433 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd"] Jul 05 00:41:28.406832 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:41:28.406805 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3654b0a_f6d2_407f_b117_a371c380df55.slice/crio-e9bf1f815f021e51e4777d28fe74e78f8d5e42364125283d77a863090ec83825 WatchSource:0}: Error finding container e9bf1f815f021e51e4777d28fe74e78f8d5e42364125283d77a863090ec83825: Status 404 returned error can't find the container with id e9bf1f815f021e51e4777d28fe74e78f8d5e42364125283d77a863090ec83825 Jul 05 00:41:29.090202 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:29.090175 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" event={"ID":"d3654b0a-f6d2-407f-b117-a371c380df55","Type":"ContainerStarted","Data":"f4292982cc6daff48aebb47d6deeae953f7da5baed26073a0115cbe1fc5f14ca"} Jul 05 00:41:29.090314 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:29.090225 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" event={"ID":"d3654b0a-f6d2-407f-b117-a371c380df55","Type":"ContainerStarted","Data":"e9bf1f815f021e51e4777d28fe74e78f8d5e42364125283d77a863090ec83825"} Jul 05 00:41:29.090314 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:29.090291 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:29.106446 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:29.106407 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" podStartSLOduration=1.49304858 podStartE2EDuration="2.106393246s" totalImagesPullingTime="613.344666ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:41:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:41:29.104654505 +0000 UTC m=+1927.041019883" watchObservedRunningTime="2026-07-05 00:41:29.106393246 +0000 UTC m=+1927.042758624" Jul 05 00:41:31.782764 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:31.782723 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:32.420100 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:32.420060 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:32.511270 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:32.511236 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:32.830620 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:32.830518 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:35.100069 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:35.100042 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:38.421608 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:38.421569 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:38.522260 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:38.522225 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:38.846512 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:38.846415 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:39.062470 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:39.062426 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd"] Jul 05 00:41:39.062734 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:39.062694 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" podUID="d3654b0a-f6d2-407f-b117-a371c380df55" containerName="maas-api" containerID="cri-o://f4292982cc6daff48aebb47d6deeae953f7da5baed26073a0115cbe1fc5f14ca" gracePeriod=30 Jul 05 00:41:39.084151 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:39.084121 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:39.308164 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:39.308139 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:39.406208 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:39.406171 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgmfp\" (UniqueName: \"kubernetes.io/projected/d3654b0a-f6d2-407f-b117-a371c380df55-kube-api-access-hgmfp\") pod \"d3654b0a-f6d2-407f-b117-a371c380df55\" (UID: \"d3654b0a-f6d2-407f-b117-a371c380df55\") " Jul 05 00:41:39.406371 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:39.406257 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3654b0a-f6d2-407f-b117-a371c380df55-maas-api-tls\") pod \"d3654b0a-f6d2-407f-b117-a371c380df55\" (UID: \"d3654b0a-f6d2-407f-b117-a371c380df55\") " Jul 05 00:41:39.406371 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:39.406306 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3654b0a-f6d2-407f-b117-a371c380df55-openshift-service-ca\") pod \"d3654b0a-f6d2-407f-b117-a371c380df55\" (UID: \"d3654b0a-f6d2-407f-b117-a371c380df55\") " Jul 05 00:41:39.406694 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:39.406667 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3654b0a-f6d2-407f-b117-a371c380df55-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d3654b0a-f6d2-407f-b117-a371c380df55" (UID: "d3654b0a-f6d2-407f-b117-a371c380df55"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:41:39.408345 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:39.408322 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3654b0a-f6d2-407f-b117-a371c380df55-kube-api-access-hgmfp" (OuterVolumeSpecName: "kube-api-access-hgmfp") pod "d3654b0a-f6d2-407f-b117-a371c380df55" (UID: "d3654b0a-f6d2-407f-b117-a371c380df55"). InnerVolumeSpecName "kube-api-access-hgmfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:41:39.408440 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:39.408416 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3654b0a-f6d2-407f-b117-a371c380df55-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d3654b0a-f6d2-407f-b117-a371c380df55" (UID: "d3654b0a-f6d2-407f-b117-a371c380df55"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 00:41:39.469593 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:39.469520 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:39.507103 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:39.507072 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgmfp\" (UniqueName: \"kubernetes.io/projected/d3654b0a-f6d2-407f-b117-a371c380df55-kube-api-access-hgmfp\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:41:39.507103 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:39.507102 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3654b0a-f6d2-407f-b117-a371c380df55-maas-api-tls\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:41:39.507103 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:39.507113 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3654b0a-f6d2-407f-b117-a371c380df55-openshift-service-ca\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:41:40.141023 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:40.140986 2558 generic.go:356] "Generic (PLEG): container finished" podID="d3654b0a-f6d2-407f-b117-a371c380df55" containerID="f4292982cc6daff48aebb47d6deeae953f7da5baed26073a0115cbe1fc5f14ca" exitCode=0 Jul 05 00:41:40.141177 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:40.141035 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" Jul 05 00:41:40.141177 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:40.141074 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" event={"ID":"d3654b0a-f6d2-407f-b117-a371c380df55","Type":"ContainerDied","Data":"f4292982cc6daff48aebb47d6deeae953f7da5baed26073a0115cbe1fc5f14ca"} Jul 05 00:41:40.141177 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:40.141129 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd" event={"ID":"d3654b0a-f6d2-407f-b117-a371c380df55","Type":"ContainerDied","Data":"e9bf1f815f021e51e4777d28fe74e78f8d5e42364125283d77a863090ec83825"} Jul 05 00:41:40.141177 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:40.141151 2558 scope.go:117] "RemoveContainer" containerID="f4292982cc6daff48aebb47d6deeae953f7da5baed26073a0115cbe1fc5f14ca" Jul 05 00:41:40.150617 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:40.150595 2558 scope.go:117] "RemoveContainer" containerID="f4292982cc6daff48aebb47d6deeae953f7da5baed26073a0115cbe1fc5f14ca" Jul 05 00:41:40.150960 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:41:40.150924 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4292982cc6daff48aebb47d6deeae953f7da5baed26073a0115cbe1fc5f14ca\": container with ID starting with f4292982cc6daff48aebb47d6deeae953f7da5baed26073a0115cbe1fc5f14ca not found: ID does not exist" containerID="f4292982cc6daff48aebb47d6deeae953f7da5baed26073a0115cbe1fc5f14ca" Jul 05 00:41:40.151060 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:40.150957 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4292982cc6daff48aebb47d6deeae953f7da5baed26073a0115cbe1fc5f14ca"} err="failed to get container status \"f4292982cc6daff48aebb47d6deeae953f7da5baed26073a0115cbe1fc5f14ca\": rpc error: code = NotFound desc = could not find container \"f4292982cc6daff48aebb47d6deeae953f7da5baed26073a0115cbe1fc5f14ca\": container with ID starting with f4292982cc6daff48aebb47d6deeae953f7da5baed26073a0115cbe1fc5f14ca not found: ID does not exist" Jul 05 00:41:40.162006 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:40.161984 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd"] Jul 05 00:41:40.164932 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:40.164912 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-125bebca-57c6bf6955-78dxd"] Jul 05 00:41:40.624098 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:40.624018 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3654b0a-f6d2-407f-b117-a371c380df55" path="/var/lib/kubelet/pods/d3654b0a-f6d2-407f-b117-a371c380df55/volumes" Jul 05 00:41:44.967063 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:44.967029 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:45.066561 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:45.066530 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:45.238604 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:45.238508 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:45.442269 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:45.442232 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:45.570415 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:45.570326 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:45.670312 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:45.670276 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:45.770015 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:45.769984 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:51.408081 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:51.408038 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:51.471919 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:51.471854 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:51.534273 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:51.534240 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:51.634137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:51.634093 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:51.735878 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:51.735790 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:57.535264 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:57.535220 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:41:57.588606 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:41:57.588577 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:03.906104 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:03.906066 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:04.195574 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:04.195491 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:04.411665 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:04.411621 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:07.511756 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.511726 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:07.589142 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.589114 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh"] Jul 05 00:42:07.589707 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.589682 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3654b0a-f6d2-407f-b117-a371c380df55" containerName="maas-api" Jul 05 00:42:07.589707 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.589710 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3654b0a-f6d2-407f-b117-a371c380df55" containerName="maas-api" Jul 05 00:42:07.589895 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.589837 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3654b0a-f6d2-407f-b117-a371c380df55" containerName="maas-api" Jul 05 00:42:07.591162 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.591134 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:07.593480 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.593461 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:42:07.596131 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.596107 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-e214bcba-serving-cert\"" Jul 05 00:42:07.605319 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.605295 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh"] Jul 05 00:42:07.650322 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.650289 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsmzv\" (UniqueName: \"kubernetes.io/projected/8cca6cfd-a896-4822-ad60-23933d3ab395-kube-api-access-dsmzv\") pod \"maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh\" (UID: \"8cca6cfd-a896-4822-ad60-23933d3ab395\") " pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:42:07.650497 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.650332 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cca6cfd-a896-4822-ad60-23933d3ab395-maas-api-tls\") pod \"maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh\" (UID: \"8cca6cfd-a896-4822-ad60-23933d3ab395\") " pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:42:07.650497 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.650352 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cca6cfd-a896-4822-ad60-23933d3ab395-openshift-service-ca\") pod \"maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh\" (UID: \"8cca6cfd-a896-4822-ad60-23933d3ab395\") " pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:42:07.653602 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.653579 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:07.751342 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.751309 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsmzv\" (UniqueName: \"kubernetes.io/projected/8cca6cfd-a896-4822-ad60-23933d3ab395-kube-api-access-dsmzv\") pod \"maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh\" (UID: \"8cca6cfd-a896-4822-ad60-23933d3ab395\") " pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:42:07.751518 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.751352 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cca6cfd-a896-4822-ad60-23933d3ab395-maas-api-tls\") pod \"maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh\" (UID: \"8cca6cfd-a896-4822-ad60-23933d3ab395\") " pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:42:07.751518 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.751373 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cca6cfd-a896-4822-ad60-23933d3ab395-openshift-service-ca\") pod \"maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh\" (UID: \"8cca6cfd-a896-4822-ad60-23933d3ab395\") " pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:42:07.751518 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:42:07.751466 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-e214bcba-serving-cert: secret "maas-api-e2e-mt-e214bcba-serving-cert" not found Jul 05 00:42:07.751645 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:42:07.751526 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cca6cfd-a896-4822-ad60-23933d3ab395-maas-api-tls podName:8cca6cfd-a896-4822-ad60-23933d3ab395 nodeName:}" failed. No retries permitted until 2026-07-05 00:42:08.251510353 +0000 UTC m=+1966.187875710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8cca6cfd-a896-4822-ad60-23933d3ab395-maas-api-tls") pod "maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" (UID: "8cca6cfd-a896-4822-ad60-23933d3ab395") : secret "maas-api-e2e-mt-e214bcba-serving-cert" not found Jul 05 00:42:07.751945 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.751929 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cca6cfd-a896-4822-ad60-23933d3ab395-openshift-service-ca\") pod \"maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh\" (UID: \"8cca6cfd-a896-4822-ad60-23933d3ab395\") " pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:42:07.759852 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:07.759827 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsmzv\" (UniqueName: \"kubernetes.io/projected/8cca6cfd-a896-4822-ad60-23933d3ab395-kube-api-access-dsmzv\") pod \"maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh\" (UID: \"8cca6cfd-a896-4822-ad60-23933d3ab395\") " pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:42:08.039817 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:08.039785 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:08.138476 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:08.138435 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:08.256244 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:08.256212 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cca6cfd-a896-4822-ad60-23933d3ab395-maas-api-tls\") pod \"maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh\" (UID: \"8cca6cfd-a896-4822-ad60-23933d3ab395\") " pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:42:08.258977 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:08.258952 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cca6cfd-a896-4822-ad60-23933d3ab395-maas-api-tls\") pod \"maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh\" (UID: \"8cca6cfd-a896-4822-ad60-23933d3ab395\") " pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:42:08.504487 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:08.504448 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:42:08.645321 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:08.645299 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh"] Jul 05 00:42:08.647623 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:42:08.647593 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cca6cfd_a896_4822_ad60_23933d3ab395.slice/crio-ec450ff56e9a8c9ee8cfd279a4ec54f8e4968a5246684444740024b398c6ac04 WatchSource:0}: Error finding container ec450ff56e9a8c9ee8cfd279a4ec54f8e4968a5246684444740024b398c6ac04: Status 404 returned error can't find the container with id ec450ff56e9a8c9ee8cfd279a4ec54f8e4968a5246684444740024b398c6ac04 Jul 05 00:42:09.087253 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:09.087209 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:09.185862 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:09.185821 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:09.295512 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:09.295410 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" event={"ID":"8cca6cfd-a896-4822-ad60-23933d3ab395","Type":"ContainerStarted","Data":"cabdfebcddfeb39658db699c8c93594bedd719b3b8f267e71a5c46523263d9e4"} Jul 05 00:42:09.295679 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:09.295515 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:42:09.295679 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:09.295540 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" event={"ID":"8cca6cfd-a896-4822-ad60-23933d3ab395","Type":"ContainerStarted","Data":"ec450ff56e9a8c9ee8cfd279a4ec54f8e4968a5246684444740024b398c6ac04"} Jul 05 00:42:09.312647 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:09.312595 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" podStartSLOduration=1.98360986 podStartE2EDuration="2.31257745s" totalImagesPullingTime="328.96759ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:42:09.311105041 +0000 UTC m=+1967.247470432" watchObservedRunningTime="2026-07-05 00:42:09.31257745 +0000 UTC m=+1967.248942825" Jul 05 00:42:09.821208 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:09.821172 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:09.919584 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:09.919554 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:12.021630 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:12.021598 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:15.305525 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:15.305455 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:42:44.108117 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:44.108085 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:42:51.866118 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:42:51.866073 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:10.092503 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.092457 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:10.293266 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.293224 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh"] Jul 05 00:43:10.293490 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.293444 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" podUID="8cca6cfd-a896-4822-ad60-23933d3ab395" containerName="maas-api" containerID="cri-o://cabdfebcddfeb39658db699c8c93594bedd719b3b8f267e71a5c46523263d9e4" gracePeriod=30 Jul 05 00:43:10.300592 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.300553 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" podUID="8cca6cfd-a896-4822-ad60-23933d3ab395" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.42:8443/health\": dial tcp 10.134.0.42:8443: connect: connection refused" Jul 05 00:43:10.321470 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.321425 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:10.544796 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.544771 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:43:10.587045 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.587009 2558 generic.go:356] "Generic (PLEG): container finished" podID="8cca6cfd-a896-4822-ad60-23933d3ab395" containerID="cabdfebcddfeb39658db699c8c93594bedd719b3b8f267e71a5c46523263d9e4" exitCode=0 Jul 05 00:43:10.587238 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.587066 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" Jul 05 00:43:10.587238 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.587092 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" event={"ID":"8cca6cfd-a896-4822-ad60-23933d3ab395","Type":"ContainerDied","Data":"cabdfebcddfeb39658db699c8c93594bedd719b3b8f267e71a5c46523263d9e4"} Jul 05 00:43:10.587238 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.587154 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh" event={"ID":"8cca6cfd-a896-4822-ad60-23933d3ab395","Type":"ContainerDied","Data":"ec450ff56e9a8c9ee8cfd279a4ec54f8e4968a5246684444740024b398c6ac04"} Jul 05 00:43:10.587238 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.587179 2558 scope.go:117] "RemoveContainer" containerID="cabdfebcddfeb39658db699c8c93594bedd719b3b8f267e71a5c46523263d9e4" Jul 05 00:43:10.596115 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.596095 2558 scope.go:117] "RemoveContainer" containerID="cabdfebcddfeb39658db699c8c93594bedd719b3b8f267e71a5c46523263d9e4" Jul 05 00:43:10.596367 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:43:10.596350 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cabdfebcddfeb39658db699c8c93594bedd719b3b8f267e71a5c46523263d9e4\": container with ID starting with cabdfebcddfeb39658db699c8c93594bedd719b3b8f267e71a5c46523263d9e4 not found: ID does not exist" containerID="cabdfebcddfeb39658db699c8c93594bedd719b3b8f267e71a5c46523263d9e4" Jul 05 00:43:10.596408 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.596377 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cabdfebcddfeb39658db699c8c93594bedd719b3b8f267e71a5c46523263d9e4"} err="failed to get container status \"cabdfebcddfeb39658db699c8c93594bedd719b3b8f267e71a5c46523263d9e4\": rpc error: code = NotFound desc = could not find container \"cabdfebcddfeb39658db699c8c93594bedd719b3b8f267e71a5c46523263d9e4\": container with ID starting with cabdfebcddfeb39658db699c8c93594bedd719b3b8f267e71a5c46523263d9e4 not found: ID does not exist" Jul 05 00:43:10.727878 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.727796 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cca6cfd-a896-4822-ad60-23933d3ab395-openshift-service-ca\") pod \"8cca6cfd-a896-4822-ad60-23933d3ab395\" (UID: \"8cca6cfd-a896-4822-ad60-23933d3ab395\") " Jul 05 00:43:10.727878 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.727851 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsmzv\" (UniqueName: \"kubernetes.io/projected/8cca6cfd-a896-4822-ad60-23933d3ab395-kube-api-access-dsmzv\") pod \"8cca6cfd-a896-4822-ad60-23933d3ab395\" (UID: \"8cca6cfd-a896-4822-ad60-23933d3ab395\") " Jul 05 00:43:10.728089 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.727952 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cca6cfd-a896-4822-ad60-23933d3ab395-maas-api-tls\") pod \"8cca6cfd-a896-4822-ad60-23933d3ab395\" (UID: \"8cca6cfd-a896-4822-ad60-23933d3ab395\") " Jul 05 00:43:10.728236 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.728214 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cca6cfd-a896-4822-ad60-23933d3ab395-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8cca6cfd-a896-4822-ad60-23933d3ab395" (UID: "8cca6cfd-a896-4822-ad60-23933d3ab395"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:43:10.730194 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.730162 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cca6cfd-a896-4822-ad60-23933d3ab395-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8cca6cfd-a896-4822-ad60-23933d3ab395" (UID: "8cca6cfd-a896-4822-ad60-23933d3ab395"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 00:43:10.730315 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.730234 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cca6cfd-a896-4822-ad60-23933d3ab395-kube-api-access-dsmzv" (OuterVolumeSpecName: "kube-api-access-dsmzv") pod "8cca6cfd-a896-4822-ad60-23933d3ab395" (UID: "8cca6cfd-a896-4822-ad60-23933d3ab395"). InnerVolumeSpecName "kube-api-access-dsmzv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:43:10.829044 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.829008 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsmzv\" (UniqueName: \"kubernetes.io/projected/8cca6cfd-a896-4822-ad60-23933d3ab395-kube-api-access-dsmzv\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:43:10.829044 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.829039 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cca6cfd-a896-4822-ad60-23933d3ab395-maas-api-tls\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:43:10.829246 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.829052 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cca6cfd-a896-4822-ad60-23933d3ab395-openshift-service-ca\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:43:10.907978 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.907948 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh"] Jul 05 00:43:10.911208 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:10.911176 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-e214bcba-55cb64bc58-wx2xh"] Jul 05 00:43:12.623922 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:12.623878 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cca6cfd-a896-4822-ad60-23933d3ab395" path="/var/lib/kubelet/pods/8cca6cfd-a896-4822-ad60-23933d3ab395/volumes" Jul 05 00:43:16.146026 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:16.145987 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:21.227400 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:21.227357 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:47.915557 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:47.915475 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:48.045146 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:48.045107 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:53.528662 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:53.528621 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:53.587589 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:53.587563 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:53.799858 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:53.799720 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:54.026204 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:54.026162 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:55.036095 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:55.036055 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:55.145078 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:55.145043 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:55.274735 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:55.274697 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:55.974488 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:55.974436 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:56.071235 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:56.071203 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:56.171869 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:56.171837 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:43:57.519810 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:43:57.519764 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:03.186743 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:03.186706 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:03.246440 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:03.246388 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:03.847505 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:03.847474 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:09.568362 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:09.568327 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:09.629949 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:09.629916 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:09.700584 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:09.700545 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:09.802430 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:09.802395 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:20.272382 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:20.272343 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:20.608115 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:20.608022 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:41.567412 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:41.567369 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:47.087402 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:47.087366 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:47.151968 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:47.151932 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:47.462954 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:47.462879 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:47.761667 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:47.761583 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:48.044585 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:48.044492 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:50.442263 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.442228 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:50.538717 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.538683 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:50.597792 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.597761 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw"] Jul 05 00:44:50.598349 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.598333 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cca6cfd-a896-4822-ad60-23933d3ab395" containerName="maas-api" Jul 05 00:44:50.598403 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.598353 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cca6cfd-a896-4822-ad60-23933d3ab395" containerName="maas-api" Jul 05 00:44:50.598487 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.598476 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cca6cfd-a896-4822-ad60-23933d3ab395" containerName="maas-api" Jul 05 00:44:50.601973 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.601952 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:44:50.604775 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.604754 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-4e7441-serving-cert\"" Jul 05 00:44:50.609721 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.609414 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hltn\" (UniqueName: \"kubernetes.io/projected/efb5945b-be95-4121-9c26-d9c61042a4a6-kube-api-access-2hltn\") pod \"maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw\" (UID: \"efb5945b-be95-4121-9c26-d9c61042a4a6\") " pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:44:50.609721 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.609455 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb5945b-be95-4121-9c26-d9c61042a4a6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw\" (UID: \"efb5945b-be95-4121-9c26-d9c61042a4a6\") " pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:44:50.609721 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.609511 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb5945b-be95-4121-9c26-d9c61042a4a6-maas-api-tls\") pod \"maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw\" (UID: \"efb5945b-be95-4121-9c26-d9c61042a4a6\") " pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:44:50.611023 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.611000 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw"] Jul 05 00:44:50.625481 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.625452 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:50.710185 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.710105 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hltn\" (UniqueName: \"kubernetes.io/projected/efb5945b-be95-4121-9c26-d9c61042a4a6-kube-api-access-2hltn\") pod \"maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw\" (UID: \"efb5945b-be95-4121-9c26-d9c61042a4a6\") " pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:44:50.710185 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.710142 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb5945b-be95-4121-9c26-d9c61042a4a6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw\" (UID: \"efb5945b-be95-4121-9c26-d9c61042a4a6\") " pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:44:50.710185 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.710177 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb5945b-be95-4121-9c26-d9c61042a4a6-maas-api-tls\") pod \"maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw\" (UID: \"efb5945b-be95-4121-9c26-d9c61042a4a6\") " pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:44:50.710420 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:44:50.710304 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-4e7441-serving-cert: secret "maas-api-e2e-inf-a-4e7441-serving-cert" not found Jul 05 00:44:50.710420 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:44:50.710366 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb5945b-be95-4121-9c26-d9c61042a4a6-maas-api-tls podName:efb5945b-be95-4121-9c26-d9c61042a4a6 nodeName:}" failed. No retries permitted until 2026-07-05 00:44:51.210350638 +0000 UTC m=+2129.146715994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/efb5945b-be95-4121-9c26-d9c61042a4a6-maas-api-tls") pod "maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" (UID: "efb5945b-be95-4121-9c26-d9c61042a4a6") : secret "maas-api-e2e-inf-a-4e7441-serving-cert" not found Jul 05 00:44:50.710791 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.710772 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb5945b-be95-4121-9c26-d9c61042a4a6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw\" (UID: \"efb5945b-be95-4121-9c26-d9c61042a4a6\") " pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:44:50.721794 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.721766 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hltn\" (UniqueName: \"kubernetes.io/projected/efb5945b-be95-4121-9c26-d9c61042a4a6-kube-api-access-2hltn\") pod \"maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw\" (UID: \"efb5945b-be95-4121-9c26-d9c61042a4a6\") " pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:44:50.884925 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:50.884861 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:51.071878 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:51.071785 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:51.172480 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:51.172451 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:51.215623 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:51.215588 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb5945b-be95-4121-9c26-d9c61042a4a6-maas-api-tls\") pod \"maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw\" (UID: \"efb5945b-be95-4121-9c26-d9c61042a4a6\") " pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:44:51.218255 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:51.218230 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb5945b-be95-4121-9c26-d9c61042a4a6-maas-api-tls\") pod \"maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw\" (UID: \"efb5945b-be95-4121-9c26-d9c61042a4a6\") " pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:44:51.513290 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:51.513258 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:44:51.655688 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:44:51.655657 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb5945b_be95_4121_9c26_d9c61042a4a6.slice/crio-bf94d726646c847b503bc0acf9ffa2801cd4592ab692f9eda79ba48510dbeaae WatchSource:0}: Error finding container bf94d726646c847b503bc0acf9ffa2801cd4592ab692f9eda79ba48510dbeaae: Status 404 returned error can't find the container with id bf94d726646c847b503bc0acf9ffa2801cd4592ab692f9eda79ba48510dbeaae Jul 05 00:44:51.657391 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:51.657373 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 00:44:51.657488 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:51.657470 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw"] Jul 05 00:44:52.075244 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:52.075211 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" event={"ID":"efb5945b-be95-4121-9c26-d9c61042a4a6","Type":"ContainerStarted","Data":"bf94d726646c847b503bc0acf9ffa2801cd4592ab692f9eda79ba48510dbeaae"} Jul 05 00:44:53.081210 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:53.081176 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" event={"ID":"efb5945b-be95-4121-9c26-d9c61042a4a6","Type":"ContainerStarted","Data":"bfea7f8f85ecc4d3beab2c238f2b6b978fa843b2ffe1d4ab14831db63fd0339c"} Jul 05 00:44:53.081644 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:53.081287 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:44:53.098538 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:53.098485 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" podStartSLOduration=2.305830401 podStartE2EDuration="3.098467753s" totalImagesPullingTime="792.637352ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:44:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:44:53.096505407 +0000 UTC m=+2131.032870798" watchObservedRunningTime="2026-07-05 00:44:53.098467753 +0000 UTC m=+2131.034833131" Jul 05 00:44:54.970872 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:54.970835 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:55.312425 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:55.312327 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:55.618040 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:55.617954 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:56.109951 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:56.109885 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:58.532673 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.532643 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:58.594758 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.594720 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:58.598675 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.598648 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb"] Jul 05 00:44:58.602415 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.602396 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:44:58.605154 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.605134 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-11db4a-serving-cert\"" Jul 05 00:44:58.614393 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.614369 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb"] Jul 05 00:44:58.684540 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.684511 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0049019b-f54f-4d9f-95f7-3c1134a70d7f-maas-api-tls\") pod \"maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb\" (UID: \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\") " pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:44:58.684540 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.684544 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8fhw\" (UniqueName: \"kubernetes.io/projected/0049019b-f54f-4d9f-95f7-3c1134a70d7f-kube-api-access-r8fhw\") pod \"maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb\" (UID: \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\") " pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:44:58.684732 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.684572 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0049019b-f54f-4d9f-95f7-3c1134a70d7f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb\" (UID: \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\") " pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:44:58.697171 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.697145 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:58.785483 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.785392 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0049019b-f54f-4d9f-95f7-3c1134a70d7f-maas-api-tls\") pod \"maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb\" (UID: \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\") " pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:44:58.785483 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.785430 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8fhw\" (UniqueName: \"kubernetes.io/projected/0049019b-f54f-4d9f-95f7-3c1134a70d7f-kube-api-access-r8fhw\") pod \"maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb\" (UID: \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\") " pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:44:58.785483 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.785474 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0049019b-f54f-4d9f-95f7-3c1134a70d7f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb\" (UID: \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\") " pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:44:58.785809 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:44:58.785541 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-11db4a-serving-cert: secret "maas-api-e2e-inf-b-11db4a-serving-cert" not found Jul 05 00:44:58.785809 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:44:58.785616 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0049019b-f54f-4d9f-95f7-3c1134a70d7f-maas-api-tls podName:0049019b-f54f-4d9f-95f7-3c1134a70d7f nodeName:}" failed. No retries permitted until 2026-07-05 00:44:59.285592863 +0000 UTC m=+2137.221958221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0049019b-f54f-4d9f-95f7-3c1134a70d7f-maas-api-tls") pod "maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" (UID: "0049019b-f54f-4d9f-95f7-3c1134a70d7f") : secret "maas-api-e2e-inf-b-11db4a-serving-cert" not found Jul 05 00:44:58.786273 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.786253 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0049019b-f54f-4d9f-95f7-3c1134a70d7f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb\" (UID: \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\") " pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:44:58.794334 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:58.794310 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8fhw\" (UniqueName: \"kubernetes.io/projected/0049019b-f54f-4d9f-95f7-3c1134a70d7f-kube-api-access-r8fhw\") pod \"maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb\" (UID: \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\") " pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:44:59.091313 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:59.091235 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:44:59.184171 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:59.184138 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:59.282027 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:59.281997 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:59.290147 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:59.290119 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0049019b-f54f-4d9f-95f7-3c1134a70d7f-maas-api-tls\") pod \"maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb\" (UID: \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\") " pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:44:59.292662 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:59.292640 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0049019b-f54f-4d9f-95f7-3c1134a70d7f-maas-api-tls\") pod \"maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb\" (UID: \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\") " pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:44:59.382581 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:59.382497 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:44:59.513366 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:59.513325 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:44:59.647365 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:44:59.647341 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb"] Jul 05 00:44:59.650577 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:44:59.650549 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0049019b_f54f_4d9f_95f7_3c1134a70d7f.slice/crio-32d6a960bf2550f2a1a0ac4987a779d7b75519b3ec1aeea9ffa17a18fa72af15 WatchSource:0}: Error finding container 32d6a960bf2550f2a1a0ac4987a779d7b75519b3ec1aeea9ffa17a18fa72af15: Status 404 returned error can't find the container with id 32d6a960bf2550f2a1a0ac4987a779d7b75519b3ec1aeea9ffa17a18fa72af15 Jul 05 00:45:00.116758 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:00.116719 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" event={"ID":"0049019b-f54f-4d9f-95f7-3c1134a70d7f","Type":"ContainerStarted","Data":"247092e94a44e9d500ffe28b0ad4cefe97e7c101315cb00647102e75477a226a"} Jul 05 00:45:00.116957 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:00.116772 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" event={"ID":"0049019b-f54f-4d9f-95f7-3c1134a70d7f","Type":"ContainerStarted","Data":"32d6a960bf2550f2a1a0ac4987a779d7b75519b3ec1aeea9ffa17a18fa72af15"} Jul 05 00:45:00.116957 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:00.116815 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:45:00.133822 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:00.133774 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" podStartSLOduration=1.836697207 podStartE2EDuration="2.133759974s" totalImagesPullingTime="297.062767ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:45:00.132610673 +0000 UTC m=+2138.068976073" watchObservedRunningTime="2026-07-05 00:45:00.133759974 +0000 UTC m=+2138.070125332" Jul 05 00:45:03.041624 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:03.041590 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:06.126589 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:06.126560 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:45:07.490744 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:07.490700 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:11.697349 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:11.697309 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:11.797188 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:11.797151 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:18.226391 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:18.226348 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:19.877446 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:19.877407 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:19.978133 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:19.978094 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:23.416555 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:23.416509 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:37.125326 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:37.125280 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:55.807800 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:55.807753 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:56.110672 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:56.110575 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:56.478460 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:56.478422 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:59.489788 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.489753 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:59.558214 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.558184 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7"] Jul 05 00:45:59.562313 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.562292 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:45:59.564860 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.564842 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-reuse-5b7734-serving-cert\"" Jul 05 00:45:59.570372 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.570347 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7"] Jul 05 00:45:59.572142 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.571757 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:59.617012 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.616978 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-openshift-service-ca\") pod \"maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7\" (UID: \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\") " pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:45:59.617175 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.617049 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t78pk\" (UniqueName: \"kubernetes.io/projected/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-kube-api-access-t78pk\") pod \"maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7\" (UID: \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\") " pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:45:59.617175 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.617096 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-maas-api-tls\") pod \"maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7\" (UID: \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\") " pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:45:59.630098 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.630072 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:59.717966 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.717931 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t78pk\" (UniqueName: \"kubernetes.io/projected/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-kube-api-access-t78pk\") pod \"maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7\" (UID: \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\") " pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:45:59.718146 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.717976 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-maas-api-tls\") pod \"maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7\" (UID: \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\") " pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:45:59.718146 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.718085 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-openshift-service-ca\") pod \"maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7\" (UID: \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\") " pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:45:59.718763 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.718730 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-openshift-service-ca\") pod \"maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7\" (UID: \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\") " pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:45:59.720690 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.720662 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-maas-api-tls\") pod \"maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7\" (UID: \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\") " pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:45:59.727188 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.727157 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t78pk\" (UniqueName: \"kubernetes.io/projected/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-kube-api-access-t78pk\") pod \"maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7\" (UID: \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\") " pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:45:59.737931 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.737713 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:45:59.876037 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:45:59.875957 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:46:00.016409 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:00.016380 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7"] Jul 05 00:46:00.019456 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:46:00.019427 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8586cdf_ea19_4c13_b1c9_f05735ff64f9.slice/crio-f4f3f3069b3f674d29759598717a667fb1a5df4830e6f39e85ead2e25b7b6098 WatchSource:0}: Error finding container f4f3f3069b3f674d29759598717a667fb1a5df4830e6f39e85ead2e25b7b6098: Status 404 returned error can't find the container with id f4f3f3069b3f674d29759598717a667fb1a5df4830e6f39e85ead2e25b7b6098 Jul 05 00:46:00.036931 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:00.036871 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:46:00.133389 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:00.133288 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:46:00.409086 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:00.409062 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" event={"ID":"b8586cdf-ea19-4c13-b1c9-f05735ff64f9","Type":"ContainerStarted","Data":"f4f3f3069b3f674d29759598717a667fb1a5df4830e6f39e85ead2e25b7b6098"} Jul 05 00:46:01.414950 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:01.414913 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" event={"ID":"b8586cdf-ea19-4c13-b1c9-f05735ff64f9","Type":"ContainerStarted","Data":"486990fd23de8e295d27c2acb8e09ddb9f2466a72fdbca2464ddc1fae29c1aa1"} Jul 05 00:46:01.415322 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:01.415020 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:46:01.432454 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:01.432410 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" podStartSLOduration=2.056559511 podStartE2EDuration="2.432397037s" totalImagesPullingTime="375.837526ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:45:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:46:01.43113596 +0000 UTC m=+2199.367501354" watchObservedRunningTime="2026-07-05 00:46:01.432397037 +0000 UTC m=+2199.368762464" Jul 05 00:46:03.901723 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:03.901691 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:46:05.322826 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:05.322785 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:46:07.430542 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:07.430513 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:46:30.824037 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:30.823992 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:46:31.137417 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.137328 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7"] Jul 05 00:46:31.137583 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.137527 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" podUID="b8586cdf-ea19-4c13-b1c9-f05735ff64f9" containerName="maas-api" containerID="cri-o://486990fd23de8e295d27c2acb8e09ddb9f2466a72fdbca2464ddc1fae29c1aa1" gracePeriod=30 Jul 05 00:46:31.159202 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.159172 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:46:31.209256 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.209222 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:46:31.401585 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.401559 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:46:31.489751 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.489714 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-openshift-service-ca\") pod \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\" (UID: \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\") " Jul 05 00:46:31.489948 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.489815 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t78pk\" (UniqueName: \"kubernetes.io/projected/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-kube-api-access-t78pk\") pod \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\" (UID: \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\") " Jul 05 00:46:31.489948 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.489842 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-maas-api-tls\") pod \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\" (UID: \"b8586cdf-ea19-4c13-b1c9-f05735ff64f9\") " Jul 05 00:46:31.490119 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.490092 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b8586cdf-ea19-4c13-b1c9-f05735ff64f9" (UID: "b8586cdf-ea19-4c13-b1c9-f05735ff64f9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:46:31.490215 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.490195 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-openshift-service-ca\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:46:31.492053 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.492030 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-kube-api-access-t78pk" (OuterVolumeSpecName: "kube-api-access-t78pk") pod "b8586cdf-ea19-4c13-b1c9-f05735ff64f9" (UID: "b8586cdf-ea19-4c13-b1c9-f05735ff64f9"). InnerVolumeSpecName "kube-api-access-t78pk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:46:31.492119 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.492088 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b8586cdf-ea19-4c13-b1c9-f05735ff64f9" (UID: "b8586cdf-ea19-4c13-b1c9-f05735ff64f9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 00:46:31.561732 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.561701 2558 generic.go:356] "Generic (PLEG): container finished" podID="b8586cdf-ea19-4c13-b1c9-f05735ff64f9" containerID="486990fd23de8e295d27c2acb8e09ddb9f2466a72fdbca2464ddc1fae29c1aa1" exitCode=0 Jul 05 00:46:31.561882 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.561743 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" Jul 05 00:46:31.561882 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.561791 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" event={"ID":"b8586cdf-ea19-4c13-b1c9-f05735ff64f9","Type":"ContainerDied","Data":"486990fd23de8e295d27c2acb8e09ddb9f2466a72fdbca2464ddc1fae29c1aa1"} Jul 05 00:46:31.562026 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.561886 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7" event={"ID":"b8586cdf-ea19-4c13-b1c9-f05735ff64f9","Type":"ContainerDied","Data":"f4f3f3069b3f674d29759598717a667fb1a5df4830e6f39e85ead2e25b7b6098"} Jul 05 00:46:31.562026 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.561937 2558 scope.go:117] "RemoveContainer" containerID="486990fd23de8e295d27c2acb8e09ddb9f2466a72fdbca2464ddc1fae29c1aa1" Jul 05 00:46:31.571083 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.571064 2558 scope.go:117] "RemoveContainer" containerID="486990fd23de8e295d27c2acb8e09ddb9f2466a72fdbca2464ddc1fae29c1aa1" Jul 05 00:46:31.571324 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:46:31.571307 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"486990fd23de8e295d27c2acb8e09ddb9f2466a72fdbca2464ddc1fae29c1aa1\": container with ID starting with 486990fd23de8e295d27c2acb8e09ddb9f2466a72fdbca2464ddc1fae29c1aa1 not found: ID does not exist" containerID="486990fd23de8e295d27c2acb8e09ddb9f2466a72fdbca2464ddc1fae29c1aa1" Jul 05 00:46:31.571384 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.571333 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"486990fd23de8e295d27c2acb8e09ddb9f2466a72fdbca2464ddc1fae29c1aa1"} err="failed to get container status \"486990fd23de8e295d27c2acb8e09ddb9f2466a72fdbca2464ddc1fae29c1aa1\": rpc error: code = NotFound desc = could not find container \"486990fd23de8e295d27c2acb8e09ddb9f2466a72fdbca2464ddc1fae29c1aa1\": container with ID starting with 486990fd23de8e295d27c2acb8e09ddb9f2466a72fdbca2464ddc1fae29c1aa1 not found: ID does not exist" Jul 05 00:46:31.582762 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.582731 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7"] Jul 05 00:46:31.586453 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.586436 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-reuse-5b7734-75bd8d7b89-5xrv7"] Jul 05 00:46:31.590856 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.590839 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t78pk\" (UniqueName: \"kubernetes.io/projected/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-kube-api-access-t78pk\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:46:31.590926 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.590860 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8586cdf-ea19-4c13-b1c9-f05735ff64f9-maas-api-tls\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:46:31.657762 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.657697 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:46:31.723062 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:31.723037 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:46:32.624459 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:32.624431 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8586cdf-ea19-4c13-b1c9-f05735ff64f9" path="/var/lib/kubelet/pods/b8586cdf-ea19-4c13-b1c9-f05735ff64f9/volumes" Jul 05 00:46:36.180208 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:46:36.180166 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:09.740540 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:09.740439 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:09.847425 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:09.847391 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:12.273510 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:12.273472 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:13.009445 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:13.009411 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:18.624095 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:18.624061 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:18.679398 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:18.679361 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:18.901407 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:18.901364 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:19.178162 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:19.178064 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:20.028614 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:20.028579 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:20.906633 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:20.906589 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:21.089367 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.089334 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:21.470660 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.470626 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:21.569952 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.569897 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:21.672739 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.672703 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw"] Jul 05 00:47:21.673079 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.672971 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" podUID="efb5945b-be95-4121-9c26-d9c61042a4a6" containerName="maas-api" containerID="cri-o://bfea7f8f85ecc4d3beab2c238f2b6b978fa843b2ffe1d4ab14831db63fd0339c" gracePeriod=30 Jul 05 00:47:21.704936 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.704863 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:47:21.803280 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.803247 2558 generic.go:356] "Generic (PLEG): container finished" podID="efb5945b-be95-4121-9c26-d9c61042a4a6" containerID="bfea7f8f85ecc4d3beab2c238f2b6b978fa843b2ffe1d4ab14831db63fd0339c" exitCode=0 Jul 05 00:47:21.803422 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.803316 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" event={"ID":"efb5945b-be95-4121-9c26-d9c61042a4a6","Type":"ContainerDied","Data":"bfea7f8f85ecc4d3beab2c238f2b6b978fa843b2ffe1d4ab14831db63fd0339c"} Jul 05 00:47:21.929495 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.929466 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:47:21.946121 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.946089 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb5945b-be95-4121-9c26-d9c61042a4a6-maas-api-tls\") pod \"efb5945b-be95-4121-9c26-d9c61042a4a6\" (UID: \"efb5945b-be95-4121-9c26-d9c61042a4a6\") " Jul 05 00:47:21.946288 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.946190 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hltn\" (UniqueName: \"kubernetes.io/projected/efb5945b-be95-4121-9c26-d9c61042a4a6-kube-api-access-2hltn\") pod \"efb5945b-be95-4121-9c26-d9c61042a4a6\" (UID: \"efb5945b-be95-4121-9c26-d9c61042a4a6\") " Jul 05 00:47:21.946288 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.946217 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb5945b-be95-4121-9c26-d9c61042a4a6-openshift-service-ca\") pod \"efb5945b-be95-4121-9c26-d9c61042a4a6\" (UID: \"efb5945b-be95-4121-9c26-d9c61042a4a6\") " Jul 05 00:47:21.946790 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.946590 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efb5945b-be95-4121-9c26-d9c61042a4a6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "efb5945b-be95-4121-9c26-d9c61042a4a6" (UID: "efb5945b-be95-4121-9c26-d9c61042a4a6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:47:21.950328 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.950303 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb5945b-be95-4121-9c26-d9c61042a4a6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "efb5945b-be95-4121-9c26-d9c61042a4a6" (UID: "efb5945b-be95-4121-9c26-d9c61042a4a6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 00:47:21.950632 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:21.950607 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efb5945b-be95-4121-9c26-d9c61042a4a6-kube-api-access-2hltn" (OuterVolumeSpecName: "kube-api-access-2hltn") pod "efb5945b-be95-4121-9c26-d9c61042a4a6" (UID: "efb5945b-be95-4121-9c26-d9c61042a4a6"). InnerVolumeSpecName "kube-api-access-2hltn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:47:22.047080 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:22.046976 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efb5945b-be95-4121-9c26-d9c61042a4a6-maas-api-tls\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:47:22.047080 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:22.047012 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hltn\" (UniqueName: \"kubernetes.io/projected/efb5945b-be95-4121-9c26-d9c61042a4a6-kube-api-access-2hltn\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:47:22.047080 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:22.047021 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efb5945b-be95-4121-9c26-d9c61042a4a6-openshift-service-ca\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:47:22.809304 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:22.809272 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" event={"ID":"efb5945b-be95-4121-9c26-d9c61042a4a6","Type":"ContainerDied","Data":"bf94d726646c847b503bc0acf9ffa2801cd4592ab692f9eda79ba48510dbeaae"} Jul 05 00:47:22.809703 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:22.809279 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw" Jul 05 00:47:22.809703 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:22.809334 2558 scope.go:117] "RemoveContainer" containerID="bfea7f8f85ecc4d3beab2c238f2b6b978fa843b2ffe1d4ab14831db63fd0339c" Jul 05 00:47:22.825387 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:22.825364 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw"] Jul 05 00:47:22.831399 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:22.831379 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-4e7441-744b7dfbdc-k95nw"] Jul 05 00:47:24.624345 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:24.624313 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efb5945b-be95-4121-9c26-d9c61042a4a6" path="/var/lib/kubelet/pods/efb5945b-be95-4121-9c26-d9c61042a4a6/volumes" Jul 05 00:47:26.604736 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:47:26.604697 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:03.298829 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:03.298790 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:03.551728 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:03.551638 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:03.789881 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:03.789851 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:03.962373 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:03.962338 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:04.333749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:04.333649 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:04.450055 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:04.450006 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:04.584870 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:04.584784 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb"] Jul 05 00:48:04.585099 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:04.585022 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" podUID="0049019b-f54f-4d9f-95f7-3c1134a70d7f" containerName="maas-api" containerID="cri-o://247092e94a44e9d500ffe28b0ad4cefe97e7c101315cb00647102e75477a226a" gracePeriod=30 Jul 05 00:48:04.612178 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:04.612150 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:04.835935 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:04.835864 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:48:04.931550 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:04.931523 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0049019b-f54f-4d9f-95f7-3c1134a70d7f-openshift-service-ca\") pod \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\" (UID: \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\") " Jul 05 00:48:04.931690 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:04.931609 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8fhw\" (UniqueName: \"kubernetes.io/projected/0049019b-f54f-4d9f-95f7-3c1134a70d7f-kube-api-access-r8fhw\") pod \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\" (UID: \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\") " Jul 05 00:48:04.931690 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:04.931662 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0049019b-f54f-4d9f-95f7-3c1134a70d7f-maas-api-tls\") pod \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\" (UID: \"0049019b-f54f-4d9f-95f7-3c1134a70d7f\") " Jul 05 00:48:04.932006 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:04.931976 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0049019b-f54f-4d9f-95f7-3c1134a70d7f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0049019b-f54f-4d9f-95f7-3c1134a70d7f" (UID: "0049019b-f54f-4d9f-95f7-3c1134a70d7f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:48:04.933871 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:04.933848 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0049019b-f54f-4d9f-95f7-3c1134a70d7f-kube-api-access-r8fhw" (OuterVolumeSpecName: "kube-api-access-r8fhw") pod "0049019b-f54f-4d9f-95f7-3c1134a70d7f" (UID: "0049019b-f54f-4d9f-95f7-3c1134a70d7f"). InnerVolumeSpecName "kube-api-access-r8fhw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:48:04.933973 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:04.933954 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0049019b-f54f-4d9f-95f7-3c1134a70d7f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0049019b-f54f-4d9f-95f7-3c1134a70d7f" (UID: "0049019b-f54f-4d9f-95f7-3c1134a70d7f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 00:48:05.014021 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:05.013985 2558 generic.go:356] "Generic (PLEG): container finished" podID="0049019b-f54f-4d9f-95f7-3c1134a70d7f" containerID="247092e94a44e9d500ffe28b0ad4cefe97e7c101315cb00647102e75477a226a" exitCode=0 Jul 05 00:48:05.014178 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:05.014032 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" Jul 05 00:48:05.014178 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:05.014072 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" event={"ID":"0049019b-f54f-4d9f-95f7-3c1134a70d7f","Type":"ContainerDied","Data":"247092e94a44e9d500ffe28b0ad4cefe97e7c101315cb00647102e75477a226a"} Jul 05 00:48:05.014178 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:05.014128 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb" event={"ID":"0049019b-f54f-4d9f-95f7-3c1134a70d7f","Type":"ContainerDied","Data":"32d6a960bf2550f2a1a0ac4987a779d7b75519b3ec1aeea9ffa17a18fa72af15"} Jul 05 00:48:05.014178 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:05.014150 2558 scope.go:117] "RemoveContainer" containerID="247092e94a44e9d500ffe28b0ad4cefe97e7c101315cb00647102e75477a226a" Jul 05 00:48:05.023123 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:05.023105 2558 scope.go:117] "RemoveContainer" containerID="247092e94a44e9d500ffe28b0ad4cefe97e7c101315cb00647102e75477a226a" Jul 05 00:48:05.023373 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:48:05.023354 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"247092e94a44e9d500ffe28b0ad4cefe97e7c101315cb00647102e75477a226a\": container with ID starting with 247092e94a44e9d500ffe28b0ad4cefe97e7c101315cb00647102e75477a226a not found: ID does not exist" containerID="247092e94a44e9d500ffe28b0ad4cefe97e7c101315cb00647102e75477a226a" Jul 05 00:48:05.023436 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:05.023386 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"247092e94a44e9d500ffe28b0ad4cefe97e7c101315cb00647102e75477a226a"} err="failed to get container status \"247092e94a44e9d500ffe28b0ad4cefe97e7c101315cb00647102e75477a226a\": rpc error: code = NotFound desc = could not find container \"247092e94a44e9d500ffe28b0ad4cefe97e7c101315cb00647102e75477a226a\": container with ID starting with 247092e94a44e9d500ffe28b0ad4cefe97e7c101315cb00647102e75477a226a not found: ID does not exist" Jul 05 00:48:05.033063 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:05.033043 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0049019b-f54f-4d9f-95f7-3c1134a70d7f-maas-api-tls\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:48:05.033063 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:05.033063 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0049019b-f54f-4d9f-95f7-3c1134a70d7f-openshift-service-ca\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:48:05.033188 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:05.033073 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8fhw\" (UniqueName: \"kubernetes.io/projected/0049019b-f54f-4d9f-95f7-3c1134a70d7f-kube-api-access-r8fhw\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:48:05.038953 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:05.038931 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb"] Jul 05 00:48:05.047363 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:05.047341 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-11db4a-5cc54849f4-vq4lb"] Jul 05 00:48:06.624451 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:06.624418 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0049019b-f54f-4d9f-95f7-3c1134a70d7f" path="/var/lib/kubelet/pods/0049019b-f54f-4d9f-95f7-3c1134a70d7f/volumes" Jul 05 00:48:10.029439 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:10.029402 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:10.128289 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:10.128238 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:10.283925 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:10.283817 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:10.505351 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:10.505313 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:10.846814 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:10.846774 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:11.138636 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:11.138542 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:11.227809 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:11.227773 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:14.038378 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.038281 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:14.091699 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.091667 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5"] Jul 05 00:48:14.092288 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.092271 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb5945b-be95-4121-9c26-d9c61042a4a6" containerName="maas-api" Jul 05 00:48:14.092343 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.092293 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb5945b-be95-4121-9c26-d9c61042a4a6" containerName="maas-api" Jul 05 00:48:14.092379 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.092342 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8586cdf-ea19-4c13-b1c9-f05735ff64f9" containerName="maas-api" Jul 05 00:48:14.092379 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.092351 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8586cdf-ea19-4c13-b1c9-f05735ff64f9" containerName="maas-api" Jul 05 00:48:14.092379 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.092364 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0049019b-f54f-4d9f-95f7-3c1134a70d7f" containerName="maas-api" Jul 05 00:48:14.092379 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.092372 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0049019b-f54f-4d9f-95f7-3c1134a70d7f" containerName="maas-api" Jul 05 00:48:14.092508 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.092456 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="efb5945b-be95-4121-9c26-d9c61042a4a6" containerName="maas-api" Jul 05 00:48:14.092508 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.092469 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0049019b-f54f-4d9f-95f7-3c1134a70d7f" containerName="maas-api" Jul 05 00:48:14.092508 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.092482 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8586cdf-ea19-4c13-b1c9-f05735ff64f9" containerName="maas-api" Jul 05 00:48:14.097428 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.097403 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:14.099974 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.099951 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-174d7d-serving-cert\"" Jul 05 00:48:14.106243 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.105655 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83a50e21-8ff3-421b-a59e-6b3df948304d-maas-api-tls\") pod \"maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5\" (UID: \"83a50e21-8ff3-421b-a59e-6b3df948304d\") " pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:14.106243 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.105691 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26ccs\" (UniqueName: \"kubernetes.io/projected/83a50e21-8ff3-421b-a59e-6b3df948304d-kube-api-access-26ccs\") pod \"maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5\" (UID: \"83a50e21-8ff3-421b-a59e-6b3df948304d\") " pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:14.106243 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.105742 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5"] Jul 05 00:48:14.106243 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.105795 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83a50e21-8ff3-421b-a59e-6b3df948304d-openshift-service-ca\") pod \"maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5\" (UID: \"83a50e21-8ff3-421b-a59e-6b3df948304d\") " pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:14.108961 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.108938 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:14.166473 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.166435 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:14.206938 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.206882 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83a50e21-8ff3-421b-a59e-6b3df948304d-maas-api-tls\") pod \"maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5\" (UID: \"83a50e21-8ff3-421b-a59e-6b3df948304d\") " pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:14.207203 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.207175 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26ccs\" (UniqueName: \"kubernetes.io/projected/83a50e21-8ff3-421b-a59e-6b3df948304d-kube-api-access-26ccs\") pod \"maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5\" (UID: \"83a50e21-8ff3-421b-a59e-6b3df948304d\") " pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:14.207346 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:48:14.207066 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-174d7d-serving-cert: secret "maas-api-e2e-api-a-174d7d-serving-cert" not found Jul 05 00:48:14.207346 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:48:14.207302 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83a50e21-8ff3-421b-a59e-6b3df948304d-maas-api-tls podName:83a50e21-8ff3-421b-a59e-6b3df948304d nodeName:}" failed. No retries permitted until 2026-07-05 00:48:14.707282494 +0000 UTC m=+2332.643647856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/83a50e21-8ff3-421b-a59e-6b3df948304d-maas-api-tls") pod "maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" (UID: "83a50e21-8ff3-421b-a59e-6b3df948304d") : secret "maas-api-e2e-api-a-174d7d-serving-cert" not found Jul 05 00:48:14.207346 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.207327 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83a50e21-8ff3-421b-a59e-6b3df948304d-openshift-service-ca\") pod \"maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5\" (UID: \"83a50e21-8ff3-421b-a59e-6b3df948304d\") " pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:14.208072 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.208053 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83a50e21-8ff3-421b-a59e-6b3df948304d-openshift-service-ca\") pod \"maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5\" (UID: \"83a50e21-8ff3-421b-a59e-6b3df948304d\") " pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:14.217105 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.217078 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26ccs\" (UniqueName: \"kubernetes.io/projected/83a50e21-8ff3-421b-a59e-6b3df948304d-kube-api-access-26ccs\") pod \"maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5\" (UID: \"83a50e21-8ff3-421b-a59e-6b3df948304d\") " pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:14.449351 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.449317 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:14.712884 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.712791 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83a50e21-8ff3-421b-a59e-6b3df948304d-maas-api-tls\") pod \"maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5\" (UID: \"83a50e21-8ff3-421b-a59e-6b3df948304d\") " pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:14.715375 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.715349 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83a50e21-8ff3-421b-a59e-6b3df948304d-maas-api-tls\") pod \"maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5\" (UID: \"83a50e21-8ff3-421b-a59e-6b3df948304d\") " pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:14.754976 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.754939 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:14.852965 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:14.852934 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:15.011441 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:15.011347 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:15.155136 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:15.155112 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5"] Jul 05 00:48:15.157572 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:48:15.157530 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83a50e21_8ff3_421b_a59e_6b3df948304d.slice/crio-5591100040d27d59811c1859bba5bc0a6a008fc0e7428b091d993d21483b03ca WatchSource:0}: Error finding container 5591100040d27d59811c1859bba5bc0a6a008fc0e7428b091d993d21483b03ca: Status 404 returned error can't find the container with id 5591100040d27d59811c1859bba5bc0a6a008fc0e7428b091d993d21483b03ca Jul 05 00:48:16.071414 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:16.071317 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" event={"ID":"83a50e21-8ff3-421b-a59e-6b3df948304d","Type":"ContainerStarted","Data":"2384189606dbfd06a0c516e7bb3f70de8deb5c4d51228bffb431a4a091430d92"} Jul 05 00:48:16.071414 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:16.071391 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" event={"ID":"83a50e21-8ff3-421b-a59e-6b3df948304d","Type":"ContainerStarted","Data":"5591100040d27d59811c1859bba5bc0a6a008fc0e7428b091d993d21483b03ca"} Jul 05 00:48:16.071639 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:16.071543 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:16.088954 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:16.088879 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" podStartSLOduration=1.536950226 podStartE2EDuration="2.088863179s" totalImagesPullingTime="551.912953ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:48:16.087004276 +0000 UTC m=+2334.023369656" watchObservedRunningTime="2026-07-05 00:48:16.088863179 +0000 UTC m=+2334.025228557" Jul 05 00:48:18.562470 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:18.562437 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:19.054304 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:19.054270 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:19.368748 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:19.368647 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:19.693728 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:19.693697 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:22.081443 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.081412 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:22.697083 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.697053 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:22.752618 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.752579 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr"] Jul 05 00:48:22.756389 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.756370 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:22.759112 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.759089 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-5e8d9c-serving-cert\"" Jul 05 00:48:22.768265 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.768205 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr"] Jul 05 00:48:22.768987 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.768965 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:22.787389 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.787359 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpsn7\" (UniqueName: \"kubernetes.io/projected/6abf4b7d-1b05-4133-811c-5afb7253efaa-kube-api-access-mpsn7\") pod \"maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr\" (UID: \"6abf4b7d-1b05-4133-811c-5afb7253efaa\") " pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:22.787535 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.787437 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6abf4b7d-1b05-4133-811c-5afb7253efaa-openshift-service-ca\") pod \"maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr\" (UID: \"6abf4b7d-1b05-4133-811c-5afb7253efaa\") " pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:22.787535 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.787508 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6abf4b7d-1b05-4133-811c-5afb7253efaa-maas-api-tls\") pod \"maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr\" (UID: \"6abf4b7d-1b05-4133-811c-5afb7253efaa\") " pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:22.834371 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.834331 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:22.888155 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.888123 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6abf4b7d-1b05-4133-811c-5afb7253efaa-maas-api-tls\") pod \"maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr\" (UID: \"6abf4b7d-1b05-4133-811c-5afb7253efaa\") " pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:22.888295 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.888171 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpsn7\" (UniqueName: \"kubernetes.io/projected/6abf4b7d-1b05-4133-811c-5afb7253efaa-kube-api-access-mpsn7\") pod \"maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr\" (UID: \"6abf4b7d-1b05-4133-811c-5afb7253efaa\") " pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:22.888295 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.888231 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6abf4b7d-1b05-4133-811c-5afb7253efaa-openshift-service-ca\") pod \"maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr\" (UID: \"6abf4b7d-1b05-4133-811c-5afb7253efaa\") " pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:22.888368 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:48:22.888295 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-5e8d9c-serving-cert: secret "maas-api-e2e-api-b-5e8d9c-serving-cert" not found Jul 05 00:48:22.888403 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:48:22.888382 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6abf4b7d-1b05-4133-811c-5afb7253efaa-maas-api-tls podName:6abf4b7d-1b05-4133-811c-5afb7253efaa nodeName:}" failed. No retries permitted until 2026-07-05 00:48:23.388361595 +0000 UTC m=+2341.324726954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6abf4b7d-1b05-4133-811c-5afb7253efaa-maas-api-tls") pod "maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" (UID: "6abf4b7d-1b05-4133-811c-5afb7253efaa") : secret "maas-api-e2e-api-b-5e8d9c-serving-cert" not found Jul 05 00:48:22.888813 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.888796 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6abf4b7d-1b05-4133-811c-5afb7253efaa-openshift-service-ca\") pod \"maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr\" (UID: \"6abf4b7d-1b05-4133-811c-5afb7253efaa\") " pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:22.916590 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:22.916558 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpsn7\" (UniqueName: \"kubernetes.io/projected/6abf4b7d-1b05-4133-811c-5afb7253efaa-kube-api-access-mpsn7\") pod \"maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr\" (UID: \"6abf4b7d-1b05-4133-811c-5afb7253efaa\") " pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:23.066529 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:23.066438 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:23.331562 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:23.331477 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:23.392482 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:23.392451 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6abf4b7d-1b05-4133-811c-5afb7253efaa-maas-api-tls\") pod \"maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr\" (UID: \"6abf4b7d-1b05-4133-811c-5afb7253efaa\") " pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:23.395209 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:23.395186 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6abf4b7d-1b05-4133-811c-5afb7253efaa-maas-api-tls\") pod \"maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr\" (UID: \"6abf4b7d-1b05-4133-811c-5afb7253efaa\") " pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:23.429233 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:23.429198 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:23.670375 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:23.670339 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:23.816147 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:23.816122 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr"] Jul 05 00:48:23.818411 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:48:23.818383 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6abf4b7d_1b05_4133_811c_5afb7253efaa.slice/crio-8cbc6ae1083d59e5d47ec8c0349f58179ba25316c47c29d7c24a9df346b53eb9 WatchSource:0}: Error finding container 8cbc6ae1083d59e5d47ec8c0349f58179ba25316c47c29d7c24a9df346b53eb9: Status 404 returned error can't find the container with id 8cbc6ae1083d59e5d47ec8c0349f58179ba25316c47c29d7c24a9df346b53eb9 Jul 05 00:48:24.113954 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:24.113830 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" event={"ID":"6abf4b7d-1b05-4133-811c-5afb7253efaa","Type":"ContainerStarted","Data":"8cbc6ae1083d59e5d47ec8c0349f58179ba25316c47c29d7c24a9df346b53eb9"} Jul 05 00:48:25.120205 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:25.120166 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" event={"ID":"6abf4b7d-1b05-4133-811c-5afb7253efaa","Type":"ContainerStarted","Data":"c47d62d59d92765a7329c61b3f7482093780d929ca876092bddfdedb52b71d66"} Jul 05 00:48:25.120573 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:25.120247 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:25.137082 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:25.137037 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" podStartSLOduration=2.73905454 podStartE2EDuration="3.137023399s" totalImagesPullingTime="397.968859ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:48:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:48:25.135053774 +0000 UTC m=+2343.071419164" watchObservedRunningTime="2026-07-05 00:48:25.137023399 +0000 UTC m=+2343.073388777" Jul 05 00:48:27.215918 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:27.215865 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:31.129860 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:31.129831 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:35.372025 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:35.371982 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:35.460273 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:35.460238 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:35.578670 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:35.578635 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5"] Jul 05 00:48:35.578991 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:35.578963 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" podUID="83a50e21-8ff3-421b-a59e-6b3df948304d" containerName="maas-api" containerID="cri-o://2384189606dbfd06a0c516e7bb3f70de8deb5c4d51228bffb431a4a091430d92" gracePeriod=30 Jul 05 00:48:35.606451 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:35.606415 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:35.838878 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:35.838847 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:35.839123 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:35.839071 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:35.913075 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:35.913044 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83a50e21-8ff3-421b-a59e-6b3df948304d-openshift-service-ca\") pod \"83a50e21-8ff3-421b-a59e-6b3df948304d\" (UID: \"83a50e21-8ff3-421b-a59e-6b3df948304d\") " Jul 05 00:48:35.913232 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:35.913101 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26ccs\" (UniqueName: \"kubernetes.io/projected/83a50e21-8ff3-421b-a59e-6b3df948304d-kube-api-access-26ccs\") pod \"83a50e21-8ff3-421b-a59e-6b3df948304d\" (UID: \"83a50e21-8ff3-421b-a59e-6b3df948304d\") " Jul 05 00:48:35.913232 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:35.913166 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83a50e21-8ff3-421b-a59e-6b3df948304d-maas-api-tls\") pod \"83a50e21-8ff3-421b-a59e-6b3df948304d\" (UID: \"83a50e21-8ff3-421b-a59e-6b3df948304d\") " Jul 05 00:48:35.913467 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:35.913443 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83a50e21-8ff3-421b-a59e-6b3df948304d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "83a50e21-8ff3-421b-a59e-6b3df948304d" (UID: "83a50e21-8ff3-421b-a59e-6b3df948304d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:48:35.915347 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:35.915327 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83a50e21-8ff3-421b-a59e-6b3df948304d-kube-api-access-26ccs" (OuterVolumeSpecName: "kube-api-access-26ccs") pod "83a50e21-8ff3-421b-a59e-6b3df948304d" (UID: "83a50e21-8ff3-421b-a59e-6b3df948304d"). InnerVolumeSpecName "kube-api-access-26ccs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:48:35.915486 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:35.915471 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83a50e21-8ff3-421b-a59e-6b3df948304d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "83a50e21-8ff3-421b-a59e-6b3df948304d" (UID: "83a50e21-8ff3-421b-a59e-6b3df948304d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 00:48:36.014728 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:36.014619 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83a50e21-8ff3-421b-a59e-6b3df948304d-openshift-service-ca\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:48:36.014728 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:36.014664 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26ccs\" (UniqueName: \"kubernetes.io/projected/83a50e21-8ff3-421b-a59e-6b3df948304d-kube-api-access-26ccs\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:48:36.014728 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:36.014676 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83a50e21-8ff3-421b-a59e-6b3df948304d-maas-api-tls\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:48:36.173626 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:36.173595 2558 generic.go:356] "Generic (PLEG): container finished" podID="83a50e21-8ff3-421b-a59e-6b3df948304d" containerID="2384189606dbfd06a0c516e7bb3f70de8deb5c4d51228bffb431a4a091430d92" exitCode=0 Jul 05 00:48:36.173626 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:36.173638 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" Jul 05 00:48:36.173626 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:36.173641 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" event={"ID":"83a50e21-8ff3-421b-a59e-6b3df948304d","Type":"ContainerDied","Data":"2384189606dbfd06a0c516e7bb3f70de8deb5c4d51228bffb431a4a091430d92"} Jul 05 00:48:36.173950 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:36.173685 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5" event={"ID":"83a50e21-8ff3-421b-a59e-6b3df948304d","Type":"ContainerDied","Data":"5591100040d27d59811c1859bba5bc0a6a008fc0e7428b091d993d21483b03ca"} Jul 05 00:48:36.173950 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:36.173705 2558 scope.go:117] "RemoveContainer" containerID="2384189606dbfd06a0c516e7bb3f70de8deb5c4d51228bffb431a4a091430d92" Jul 05 00:48:36.183066 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:36.183046 2558 scope.go:117] "RemoveContainer" containerID="2384189606dbfd06a0c516e7bb3f70de8deb5c4d51228bffb431a4a091430d92" Jul 05 00:48:36.183352 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:48:36.183332 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2384189606dbfd06a0c516e7bb3f70de8deb5c4d51228bffb431a4a091430d92\": container with ID starting with 2384189606dbfd06a0c516e7bb3f70de8deb5c4d51228bffb431a4a091430d92 not found: ID does not exist" containerID="2384189606dbfd06a0c516e7bb3f70de8deb5c4d51228bffb431a4a091430d92" Jul 05 00:48:36.183387 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:36.183361 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2384189606dbfd06a0c516e7bb3f70de8deb5c4d51228bffb431a4a091430d92"} err="failed to get container status \"2384189606dbfd06a0c516e7bb3f70de8deb5c4d51228bffb431a4a091430d92\": rpc error: code = NotFound desc = could not find container \"2384189606dbfd06a0c516e7bb3f70de8deb5c4d51228bffb431a4a091430d92\": container with ID starting with 2384189606dbfd06a0c516e7bb3f70de8deb5c4d51228bffb431a4a091430d92 not found: ID does not exist" Jul 05 00:48:36.194813 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:36.194740 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5"] Jul 05 00:48:36.196708 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:36.196685 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-174d7d-6495899dc6-qg6t5"] Jul 05 00:48:36.624167 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:36.624135 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83a50e21-8ff3-421b-a59e-6b3df948304d" path="/var/lib/kubelet/pods/83a50e21-8ff3-421b-a59e-6b3df948304d/volumes" Jul 05 00:48:41.439352 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:41.439314 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:41.538081 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:41.538049 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:41.699205 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:41.699119 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:41.915374 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:41.915344 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:42.264341 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:42.264304 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:42.550214 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:42.550119 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:42.670518 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:42.670482 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:42.771360 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:42.771326 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:42.887204 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:42.887117 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr"] Jul 05 00:48:42.887380 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:42.887354 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" podUID="6abf4b7d-1b05-4133-811c-5afb7253efaa" containerName="maas-api" containerID="cri-o://c47d62d59d92765a7329c61b3f7482093780d929ca876092bddfdedb52b71d66" gracePeriod=30 Jul 05 00:48:42.912002 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:42.911971 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:43.141585 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.141563 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:43.209173 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.209134 2558 generic.go:356] "Generic (PLEG): container finished" podID="6abf4b7d-1b05-4133-811c-5afb7253efaa" containerID="c47d62d59d92765a7329c61b3f7482093780d929ca876092bddfdedb52b71d66" exitCode=0 Jul 05 00:48:43.209372 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.209200 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" Jul 05 00:48:43.209372 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.209227 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" event={"ID":"6abf4b7d-1b05-4133-811c-5afb7253efaa","Type":"ContainerDied","Data":"c47d62d59d92765a7329c61b3f7482093780d929ca876092bddfdedb52b71d66"} Jul 05 00:48:43.209372 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.209315 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr" event={"ID":"6abf4b7d-1b05-4133-811c-5afb7253efaa","Type":"ContainerDied","Data":"8cbc6ae1083d59e5d47ec8c0349f58179ba25316c47c29d7c24a9df346b53eb9"} Jul 05 00:48:43.209372 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.209343 2558 scope.go:117] "RemoveContainer" containerID="c47d62d59d92765a7329c61b3f7482093780d929ca876092bddfdedb52b71d66" Jul 05 00:48:43.217725 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.217705 2558 scope.go:117] "RemoveContainer" containerID="c47d62d59d92765a7329c61b3f7482093780d929ca876092bddfdedb52b71d66" Jul 05 00:48:43.218025 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:48:43.218005 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c47d62d59d92765a7329c61b3f7482093780d929ca876092bddfdedb52b71d66\": container with ID starting with c47d62d59d92765a7329c61b3f7482093780d929ca876092bddfdedb52b71d66 not found: ID does not exist" containerID="c47d62d59d92765a7329c61b3f7482093780d929ca876092bddfdedb52b71d66" Jul 05 00:48:43.218102 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.218037 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47d62d59d92765a7329c61b3f7482093780d929ca876092bddfdedb52b71d66"} err="failed to get container status \"c47d62d59d92765a7329c61b3f7482093780d929ca876092bddfdedb52b71d66\": rpc error: code = NotFound desc = could not find container \"c47d62d59d92765a7329c61b3f7482093780d929ca876092bddfdedb52b71d66\": container with ID starting with c47d62d59d92765a7329c61b3f7482093780d929ca876092bddfdedb52b71d66 not found: ID does not exist" Jul 05 00:48:43.282857 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.282821 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6abf4b7d-1b05-4133-811c-5afb7253efaa-openshift-service-ca\") pod \"6abf4b7d-1b05-4133-811c-5afb7253efaa\" (UID: \"6abf4b7d-1b05-4133-811c-5afb7253efaa\") " Jul 05 00:48:43.283082 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.282914 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6abf4b7d-1b05-4133-811c-5afb7253efaa-maas-api-tls\") pod \"6abf4b7d-1b05-4133-811c-5afb7253efaa\" (UID: \"6abf4b7d-1b05-4133-811c-5afb7253efaa\") " Jul 05 00:48:43.283082 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.282942 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpsn7\" (UniqueName: \"kubernetes.io/projected/6abf4b7d-1b05-4133-811c-5afb7253efaa-kube-api-access-mpsn7\") pod \"6abf4b7d-1b05-4133-811c-5afb7253efaa\" (UID: \"6abf4b7d-1b05-4133-811c-5afb7253efaa\") " Jul 05 00:48:43.283199 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.283179 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6abf4b7d-1b05-4133-811c-5afb7253efaa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6abf4b7d-1b05-4133-811c-5afb7253efaa" (UID: "6abf4b7d-1b05-4133-811c-5afb7253efaa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:48:43.285235 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.285207 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6abf4b7d-1b05-4133-811c-5afb7253efaa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6abf4b7d-1b05-4133-811c-5afb7253efaa" (UID: "6abf4b7d-1b05-4133-811c-5afb7253efaa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 00:48:43.285235 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.285222 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6abf4b7d-1b05-4133-811c-5afb7253efaa-kube-api-access-mpsn7" (OuterVolumeSpecName: "kube-api-access-mpsn7") pod "6abf4b7d-1b05-4133-811c-5afb7253efaa" (UID: "6abf4b7d-1b05-4133-811c-5afb7253efaa"). InnerVolumeSpecName "kube-api-access-mpsn7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:48:43.384529 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.384494 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6abf4b7d-1b05-4133-811c-5afb7253efaa-maas-api-tls\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:48:43.384529 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.384520 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpsn7\" (UniqueName: \"kubernetes.io/projected/6abf4b7d-1b05-4133-811c-5afb7253efaa-kube-api-access-mpsn7\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:48:43.384529 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.384532 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6abf4b7d-1b05-4133-811c-5afb7253efaa-openshift-service-ca\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:48:43.530646 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.530619 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr"] Jul 05 00:48:43.536025 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:43.536001 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-5e8d9c-6cd5bd4f94-dmvcr"] Jul 05 00:48:44.624251 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:44.624219 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6abf4b7d-1b05-4133-811c-5afb7253efaa" path="/var/lib/kubelet/pods/6abf4b7d-1b05-4133-811c-5afb7253efaa/volumes" Jul 05 00:48:47.381066 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:47.381030 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:47.478474 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:47.478441 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:48.252330 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:48.252299 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:48.352943 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:48.352889 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:48.527848 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:48.527752 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:48.744824 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:48.744787 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:49.083144 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:49.083084 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:49.400030 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:49.399997 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:49.501550 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:49.501508 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:52.208791 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.208726 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:52.266212 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.266177 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd"] Jul 05 00:48:52.266633 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.266616 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83a50e21-8ff3-421b-a59e-6b3df948304d" containerName="maas-api" Jul 05 00:48:52.266753 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.266636 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="83a50e21-8ff3-421b-a59e-6b3df948304d" containerName="maas-api" Jul 05 00:48:52.266753 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.266649 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6abf4b7d-1b05-4133-811c-5afb7253efaa" containerName="maas-api" Jul 05 00:48:52.266753 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.266657 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6abf4b7d-1b05-4133-811c-5afb7253efaa" containerName="maas-api" Jul 05 00:48:52.266953 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.266784 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="83a50e21-8ff3-421b-a59e-6b3df948304d" containerName="maas-api" Jul 05 00:48:52.266953 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.266799 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6abf4b7d-1b05-4133-811c-5afb7253efaa" containerName="maas-api" Jul 05 00:48:52.271277 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.271257 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:48:52.273708 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.273687 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-3575e1-serving-cert\"" Jul 05 00:48:52.281401 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.281373 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd"] Jul 05 00:48:52.285022 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.284999 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:52.356993 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.356955 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:52.368585 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.368560 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-maas-api-tls\") pod \"maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd\" (UID: \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\") " pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:48:52.368709 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.368608 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd\" (UID: \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\") " pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:48:52.368709 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.368626 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr6lb\" (UniqueName: \"kubernetes.io/projected/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-kube-api-access-cr6lb\") pod \"maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd\" (UID: \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\") " pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:48:52.469384 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.469297 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-maas-api-tls\") pod \"maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd\" (UID: \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\") " pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:48:52.469384 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.469350 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd\" (UID: \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\") " pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:48:52.469384 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.469385 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr6lb\" (UniqueName: \"kubernetes.io/projected/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-kube-api-access-cr6lb\") pod \"maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd\" (UID: \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\") " pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:48:52.469647 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:48:52.469428 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-3575e1-serving-cert: secret "maas-api-e2e-shared-a-3575e1-serving-cert" not found Jul 05 00:48:52.469647 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:48:52.469502 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-maas-api-tls podName:a124ae1b-a5b0-4e08-a82c-3a91eccf5864 nodeName:}" failed. No retries permitted until 2026-07-05 00:48:52.969485265 +0000 UTC m=+2370.905850625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-maas-api-tls") pod "maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" (UID: "a124ae1b-a5b0-4e08-a82c-3a91eccf5864") : secret "maas-api-e2e-shared-a-3575e1-serving-cert" not found Jul 05 00:48:52.470135 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.470113 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd\" (UID: \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\") " pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:48:52.480367 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.480340 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr6lb\" (UniqueName: \"kubernetes.io/projected/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-kube-api-access-cr6lb\") pod \"maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd\" (UID: \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\") " pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:48:52.848066 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.847979 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:52.949061 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.949026 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:52.973853 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.973824 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-maas-api-tls\") pod \"maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd\" (UID: \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\") " pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:48:52.976241 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:52.976220 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-maas-api-tls\") pod \"maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd\" (UID: \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\") " pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:48:53.048949 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:53.048883 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:53.182589 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:53.182554 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:48:53.320360 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:53.320332 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd"] Jul 05 00:48:53.322542 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:48:53.322515 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda124ae1b_a5b0_4e08_a82c_3a91eccf5864.slice/crio-a1c3b327055d7c87c4e3be1d2a96d6a14d5ed9bdd8fdea4fd3dc47ee726d206b WatchSource:0}: Error finding container a1c3b327055d7c87c4e3be1d2a96d6a14d5ed9bdd8fdea4fd3dc47ee726d206b: Status 404 returned error can't find the container with id a1c3b327055d7c87c4e3be1d2a96d6a14d5ed9bdd8fdea4fd3dc47ee726d206b Jul 05 00:48:54.263143 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:54.263101 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" event={"ID":"a124ae1b-a5b0-4e08-a82c-3a91eccf5864","Type":"ContainerStarted","Data":"da6f338c8262a34b9bc9e4b98f5023cd5040d5435a97b2dd6f9067ff106bb713"} Jul 05 00:48:54.263325 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:54.263168 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:48:54.263325 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:54.263192 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" event={"ID":"a124ae1b-a5b0-4e08-a82c-3a91eccf5864","Type":"ContainerStarted","Data":"a1c3b327055d7c87c4e3be1d2a96d6a14d5ed9bdd8fdea4fd3dc47ee726d206b"} Jul 05 00:48:54.278295 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:54.278242 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" podStartSLOduration=1.9411233399999999 podStartE2EDuration="2.278224407s" totalImagesPullingTime="337.101067ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:48:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:48:54.277980545 +0000 UTC m=+2372.214345925" watchObservedRunningTime="2026-07-05 00:48:54.278224407 +0000 UTC m=+2372.214589789" Jul 05 00:48:56.759120 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:56.759070 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:57.094939 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:57.094810 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:57.384697 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:57.384605 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:57.847951 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:57.847887 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:48:59.907148 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:48:59.907114 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:49:00.000725 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.000681 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:49:00.005009 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.004984 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22"] Jul 05 00:49:00.008567 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.008551 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:49:00.011098 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.011081 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-1a1f49-serving-cert\"" Jul 05 00:49:00.018181 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.018161 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22"] Jul 05 00:49:00.025513 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.025490 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e41886d9-e354-4364-9212-044fff3c9b61-maas-api-tls\") pod \"maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22\" (UID: \"e41886d9-e354-4364-9212-044fff3c9b61\") " pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:49:00.025620 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.025576 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e41886d9-e354-4364-9212-044fff3c9b61-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22\" (UID: \"e41886d9-e354-4364-9212-044fff3c9b61\") " pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:49:00.025620 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.025615 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s6bj\" (UniqueName: \"kubernetes.io/projected/e41886d9-e354-4364-9212-044fff3c9b61-kube-api-access-8s6bj\") pod \"maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22\" (UID: \"e41886d9-e354-4364-9212-044fff3c9b61\") " pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:49:00.101701 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.101636 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:49:00.126847 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.126817 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e41886d9-e354-4364-9212-044fff3c9b61-maas-api-tls\") pod \"maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22\" (UID: \"e41886d9-e354-4364-9212-044fff3c9b61\") " pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:49:00.126987 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.126861 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e41886d9-e354-4364-9212-044fff3c9b61-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22\" (UID: \"e41886d9-e354-4364-9212-044fff3c9b61\") " pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:49:00.126987 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.126880 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s6bj\" (UniqueName: \"kubernetes.io/projected/e41886d9-e354-4364-9212-044fff3c9b61-kube-api-access-8s6bj\") pod \"maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22\" (UID: \"e41886d9-e354-4364-9212-044fff3c9b61\") " pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:49:00.127085 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:49:00.126993 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-1a1f49-serving-cert: secret "maas-api-e2e-shared-b-1a1f49-serving-cert" not found Jul 05 00:49:00.127085 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:49:00.127068 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e41886d9-e354-4364-9212-044fff3c9b61-maas-api-tls podName:e41886d9-e354-4364-9212-044fff3c9b61 nodeName:}" failed. No retries permitted until 2026-07-05 00:49:00.627045528 +0000 UTC m=+2378.563410886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e41886d9-e354-4364-9212-044fff3c9b61-maas-api-tls") pod "maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" (UID: "e41886d9-e354-4364-9212-044fff3c9b61") : secret "maas-api-e2e-shared-b-1a1f49-serving-cert" not found Jul 05 00:49:00.127533 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.127514 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e41886d9-e354-4364-9212-044fff3c9b61-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22\" (UID: \"e41886d9-e354-4364-9212-044fff3c9b61\") " pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:49:00.135333 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.135306 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s6bj\" (UniqueName: \"kubernetes.io/projected/e41886d9-e354-4364-9212-044fff3c9b61-kube-api-access-8s6bj\") pod \"maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22\" (UID: \"e41886d9-e354-4364-9212-044fff3c9b61\") " pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:49:00.273911 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.273820 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:49:00.608167 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.608071 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:49:00.631137 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.631104 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e41886d9-e354-4364-9212-044fff3c9b61-maas-api-tls\") pod \"maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22\" (UID: \"e41886d9-e354-4364-9212-044fff3c9b61\") " pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:49:00.633505 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.633482 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e41886d9-e354-4364-9212-044fff3c9b61-maas-api-tls\") pod \"maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22\" (UID: \"e41886d9-e354-4364-9212-044fff3c9b61\") " pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:49:00.728962 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.728925 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:49:00.829349 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.829320 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:49:00.919746 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:00.919715 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:49:01.087813 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:01.087784 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22"] Jul 05 00:49:01.089645 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:49:01.089620 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode41886d9_e354_4364_9212_044fff3c9b61.slice/crio-84a94acc7627cb18507fd5185937d019e972a1d245b0843cd060a0d94e07cc4f WatchSource:0}: Error finding container 84a94acc7627cb18507fd5185937d019e972a1d245b0843cd060a0d94e07cc4f: Status 404 returned error can't find the container with id 84a94acc7627cb18507fd5185937d019e972a1d245b0843cd060a0d94e07cc4f Jul 05 00:49:01.297499 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:01.297413 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" event={"ID":"e41886d9-e354-4364-9212-044fff3c9b61","Type":"ContainerStarted","Data":"84a94acc7627cb18507fd5185937d019e972a1d245b0843cd060a0d94e07cc4f"} Jul 05 00:49:02.303476 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:02.303446 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" event={"ID":"e41886d9-e354-4364-9212-044fff3c9b61","Type":"ContainerStarted","Data":"fb97a147416f8a04806f447ba3d683fb81cee418a8946d7bfca15f620e8a26ce"} Jul 05 00:49:02.303950 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:02.303525 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:49:02.320486 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:02.320432 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" podStartSLOduration=2.933482036 podStartE2EDuration="3.320415548s" totalImagesPullingTime="386.933512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 00:48:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:49:02.31814759 +0000 UTC m=+2380.254512986" watchObservedRunningTime="2026-07-05 00:49:02.320415548 +0000 UTC m=+2380.256780926" Jul 05 00:49:04.408717 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:04.408679 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:49:08.314299 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:08.314268 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:49:13.553493 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:13.553449 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:49:13.653936 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:13.653893 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:49:21.358459 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:21.358419 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:49:21.459656 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:21.459621 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:49:41.036464 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:49:41.036432 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:50:05.330069 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:50:05.329989 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:50:53.129193 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:50:53.129162 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:51:54.474322 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:51:54.474288 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:51:55.423684 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:51:55.423648 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:52:06.434674 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:52:06.434637 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:52:10.393056 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:52:10.393014 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:52:10.493705 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:52:10.493652 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:52:36.423165 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:52:36.423132 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:52:45.989746 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:52:45.989653 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:52:46.090437 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:52:46.090405 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:52:56.877084 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:52:56.877035 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:53:22.980231 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:53:22.980196 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:53:23.929875 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:53:23.929840 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:53:28.039738 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:53:28.039702 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:53:56.882983 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:53:56.882946 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:14.162173 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:14.162078 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:14.262857 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:14.262820 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:21.989302 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:21.989265 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:22.087363 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:22.087333 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:40.507147 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:40.507109 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:41.306870 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:41.306831 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:46.956838 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:46.956800 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:47.359225 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:47.359106 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:47.459334 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:47.459300 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:47.560597 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:47.560544 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:47.634428 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:47.634350 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd"] Jul 05 00:54:47.634615 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:47.634586 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" podUID="a124ae1b-a5b0-4e08-a82c-3a91eccf5864" containerName="maas-api" containerID="cri-o://da6f338c8262a34b9bc9e4b98f5023cd5040d5435a97b2dd6f9067ff106bb713" gracePeriod=30 Jul 05 00:54:47.654113 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:47.654082 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:47.888713 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:47.888693 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:54:47.944678 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:47.944644 2558 generic.go:356] "Generic (PLEG): container finished" podID="a124ae1b-a5b0-4e08-a82c-3a91eccf5864" containerID="da6f338c8262a34b9bc9e4b98f5023cd5040d5435a97b2dd6f9067ff106bb713" exitCode=0 Jul 05 00:54:47.944813 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:47.944698 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" Jul 05 00:54:47.944813 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:47.944723 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" event={"ID":"a124ae1b-a5b0-4e08-a82c-3a91eccf5864","Type":"ContainerDied","Data":"da6f338c8262a34b9bc9e4b98f5023cd5040d5435a97b2dd6f9067ff106bb713"} Jul 05 00:54:47.944813 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:47.944774 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd" event={"ID":"a124ae1b-a5b0-4e08-a82c-3a91eccf5864","Type":"ContainerDied","Data":"a1c3b327055d7c87c4e3be1d2a96d6a14d5ed9bdd8fdea4fd3dc47ee726d206b"} Jul 05 00:54:47.944813 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:47.944796 2558 scope.go:117] "RemoveContainer" containerID="da6f338c8262a34b9bc9e4b98f5023cd5040d5435a97b2dd6f9067ff106bb713" Jul 05 00:54:47.953152 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:47.953135 2558 scope.go:117] "RemoveContainer" containerID="da6f338c8262a34b9bc9e4b98f5023cd5040d5435a97b2dd6f9067ff106bb713" Jul 05 00:54:47.953385 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:54:47.953368 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da6f338c8262a34b9bc9e4b98f5023cd5040d5435a97b2dd6f9067ff106bb713\": container with ID starting with da6f338c8262a34b9bc9e4b98f5023cd5040d5435a97b2dd6f9067ff106bb713 not found: ID does not exist" containerID="da6f338c8262a34b9bc9e4b98f5023cd5040d5435a97b2dd6f9067ff106bb713" Jul 05 00:54:47.953444 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:47.953394 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da6f338c8262a34b9bc9e4b98f5023cd5040d5435a97b2dd6f9067ff106bb713"} err="failed to get container status \"da6f338c8262a34b9bc9e4b98f5023cd5040d5435a97b2dd6f9067ff106bb713\": rpc error: code = NotFound desc = could not find container \"da6f338c8262a34b9bc9e4b98f5023cd5040d5435a97b2dd6f9067ff106bb713\": container with ID starting with da6f338c8262a34b9bc9e4b98f5023cd5040d5435a97b2dd6f9067ff106bb713 not found: ID does not exist" Jul 05 00:54:48.031133 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:48.031109 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-maas-api-tls\") pod \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\" (UID: \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\") " Jul 05 00:54:48.031526 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:48.031146 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cr6lb\" (UniqueName: \"kubernetes.io/projected/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-kube-api-access-cr6lb\") pod \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\" (UID: \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\") " Jul 05 00:54:48.031526 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:48.031199 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-openshift-service-ca\") pod \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\" (UID: \"a124ae1b-a5b0-4e08-a82c-3a91eccf5864\") " Jul 05 00:54:48.031606 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:48.031583 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a124ae1b-a5b0-4e08-a82c-3a91eccf5864" (UID: "a124ae1b-a5b0-4e08-a82c-3a91eccf5864"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:54:48.033358 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:48.033331 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-kube-api-access-cr6lb" (OuterVolumeSpecName: "kube-api-access-cr6lb") pod "a124ae1b-a5b0-4e08-a82c-3a91eccf5864" (UID: "a124ae1b-a5b0-4e08-a82c-3a91eccf5864"). InnerVolumeSpecName "kube-api-access-cr6lb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:54:48.033358 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:48.033339 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a124ae1b-a5b0-4e08-a82c-3a91eccf5864" (UID: "a124ae1b-a5b0-4e08-a82c-3a91eccf5864"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 00:54:48.132029 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:48.131992 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-maas-api-tls\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:54:48.132029 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:48.132023 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cr6lb\" (UniqueName: \"kubernetes.io/projected/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-kube-api-access-cr6lb\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:54:48.132029 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:48.132034 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a124ae1b-a5b0-4e08-a82c-3a91eccf5864-openshift-service-ca\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:54:48.265595 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:48.265560 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd"] Jul 05 00:54:48.267505 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:48.267480 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-3575e1-66bf8cdf86-v67dd"] Jul 05 00:54:48.624785 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:48.624708 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a124ae1b-a5b0-4e08-a82c-3a91eccf5864" path="/var/lib/kubelet/pods/a124ae1b-a5b0-4e08-a82c-3a91eccf5864/volumes" Jul 05 00:54:52.435661 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:52.435625 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:58.196283 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:58.196253 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:58.482740 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:58.482644 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:58.743374 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:58.743297 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:58.905520 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:58.905483 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:59.285662 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.285627 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:59.383164 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.383122 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:59.476607 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.476570 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22"] Jul 05 00:54:59.476926 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.476872 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" podUID="e41886d9-e354-4364-9212-044fff3c9b61" containerName="maas-api" containerID="cri-o://fb97a147416f8a04806f447ba3d683fb81cee418a8946d7bfca15f620e8a26ce" gracePeriod=30 Jul 05 00:54:59.503433 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.503408 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:54:59.730895 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.730866 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:54:59.834718 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.834685 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8s6bj\" (UniqueName: \"kubernetes.io/projected/e41886d9-e354-4364-9212-044fff3c9b61-kube-api-access-8s6bj\") pod \"e41886d9-e354-4364-9212-044fff3c9b61\" (UID: \"e41886d9-e354-4364-9212-044fff3c9b61\") " Jul 05 00:54:59.834887 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.834763 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e41886d9-e354-4364-9212-044fff3c9b61-maas-api-tls\") pod \"e41886d9-e354-4364-9212-044fff3c9b61\" (UID: \"e41886d9-e354-4364-9212-044fff3c9b61\") " Jul 05 00:54:59.834887 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.834785 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e41886d9-e354-4364-9212-044fff3c9b61-openshift-service-ca\") pod \"e41886d9-e354-4364-9212-044fff3c9b61\" (UID: \"e41886d9-e354-4364-9212-044fff3c9b61\") " Jul 05 00:54:59.835213 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.835191 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e41886d9-e354-4364-9212-044fff3c9b61-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e41886d9-e354-4364-9212-044fff3c9b61" (UID: "e41886d9-e354-4364-9212-044fff3c9b61"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 00:54:59.836849 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.836811 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e41886d9-e354-4364-9212-044fff3c9b61-kube-api-access-8s6bj" (OuterVolumeSpecName: "kube-api-access-8s6bj") pod "e41886d9-e354-4364-9212-044fff3c9b61" (UID: "e41886d9-e354-4364-9212-044fff3c9b61"). InnerVolumeSpecName "kube-api-access-8s6bj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 00:54:59.836957 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.836871 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e41886d9-e354-4364-9212-044fff3c9b61-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e41886d9-e354-4364-9212-044fff3c9b61" (UID: "e41886d9-e354-4364-9212-044fff3c9b61"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 00:54:59.936166 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.936138 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e41886d9-e354-4364-9212-044fff3c9b61-maas-api-tls\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:54:59.936166 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.936164 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e41886d9-e354-4364-9212-044fff3c9b61-openshift-service-ca\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:54:59.936344 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:54:59.936175 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s6bj\" (UniqueName: \"kubernetes.io/projected/e41886d9-e354-4364-9212-044fff3c9b61-kube-api-access-8s6bj\") on node \"ip-10-0-130-0.ec2.internal\" DevicePath \"\"" Jul 05 00:55:00.001229 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:00.001200 2558 generic.go:356] "Generic (PLEG): container finished" podID="e41886d9-e354-4364-9212-044fff3c9b61" containerID="fb97a147416f8a04806f447ba3d683fb81cee418a8946d7bfca15f620e8a26ce" exitCode=0 Jul 05 00:55:00.001369 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:00.001249 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" Jul 05 00:55:00.001369 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:00.001288 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" event={"ID":"e41886d9-e354-4364-9212-044fff3c9b61","Type":"ContainerDied","Data":"fb97a147416f8a04806f447ba3d683fb81cee418a8946d7bfca15f620e8a26ce"} Jul 05 00:55:00.001369 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:00.001339 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22" event={"ID":"e41886d9-e354-4364-9212-044fff3c9b61","Type":"ContainerDied","Data":"84a94acc7627cb18507fd5185937d019e972a1d245b0843cd060a0d94e07cc4f"} Jul 05 00:55:00.001369 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:00.001361 2558 scope.go:117] "RemoveContainer" containerID="fb97a147416f8a04806f447ba3d683fb81cee418a8946d7bfca15f620e8a26ce" Jul 05 00:55:00.010749 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:00.010733 2558 scope.go:117] "RemoveContainer" containerID="fb97a147416f8a04806f447ba3d683fb81cee418a8946d7bfca15f620e8a26ce" Jul 05 00:55:00.010993 ip-10-0-130-0 kubenswrapper[2558]: E0705 00:55:00.010977 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb97a147416f8a04806f447ba3d683fb81cee418a8946d7bfca15f620e8a26ce\": container with ID starting with fb97a147416f8a04806f447ba3d683fb81cee418a8946d7bfca15f620e8a26ce not found: ID does not exist" containerID="fb97a147416f8a04806f447ba3d683fb81cee418a8946d7bfca15f620e8a26ce" Jul 05 00:55:00.011056 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:00.011002 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb97a147416f8a04806f447ba3d683fb81cee418a8946d7bfca15f620e8a26ce"} err="failed to get container status \"fb97a147416f8a04806f447ba3d683fb81cee418a8946d7bfca15f620e8a26ce\": rpc error: code = NotFound desc = could not find container \"fb97a147416f8a04806f447ba3d683fb81cee418a8946d7bfca15f620e8a26ce\": container with ID starting with fb97a147416f8a04806f447ba3d683fb81cee418a8946d7bfca15f620e8a26ce not found: ID does not exist" Jul 05 00:55:00.021890 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:00.021863 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22"] Jul 05 00:55:00.023615 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:00.023597 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-1a1f49-5cbc7495b7-mdt22"] Jul 05 00:55:00.624245 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:00.624211 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e41886d9-e354-4364-9212-044fff3c9b61" path="/var/lib/kubelet/pods/e41886d9-e354-4364-9212-044fff3c9b61/volumes" Jul 05 00:55:05.041670 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:05.041621 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:55:05.164883 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:05.164848 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:55:05.332892 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:05.332804 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:55:05.554499 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:05.554465 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:55:11.120513 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:11.120481 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-79bvr_a25ed367-e43c-4191-86d6-557cb7f14e4a/manager/0.log" Jul 05 00:55:11.268304 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:11.268279 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vgkg6_3e9b555f-2665-44e7-95b2-9587f4863760/manager/0.log" Jul 05 00:55:11.408008 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:11.407957 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5b96d4fbd4-5b8hr_c1f82e5b-362d-485e-980e-23891749ff79/maas-api/0.log" Jul 05 00:55:11.804556 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:11.804457 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-98468dc4b-db98h_ae82ae7b-1baf-4c76-b71f-5f3efef3df37/manager/0.log" Jul 05 00:55:11.934959 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:11.934897 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bm469_10e57536-ef8d-4e8c-a817-907de7fe3366/server/0.log" Jul 05 00:55:12.073342 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:12.073270 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dzcz4_28b5b72c-0953-4c98-a9a1-d45eab3be5d3/manager/0.log" Jul 05 00:55:12.190010 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:12.189982 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674c4bd948-84wl8_8a6ad246-4f69-4fbf-9cd2-fed27a05502e/copy-manifests/0.log" Jul 05 00:55:12.196024 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:12.195976 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674c4bd948-84wl8_8a6ad246-4f69-4fbf-9cd2-fed27a05502e/manager/0.log" Jul 05 00:55:12.604291 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:12.604261 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-8cpxr_6da95d6e-6b88-4f8b-9472-f44ab80723cc/postgres/0.log" Jul 05 00:55:14.203557 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:14.203527 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-cv5tv_93a20e30-fa1f-4676-bbfe-ec0005381944/manager/0.log" Jul 05 00:55:15.367884 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:15.367850 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fsbtc_50249fa7-6685-4f7d-9ba2-feed1eedcbf0/discovery/0.log" Jul 05 00:55:15.629455 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:15.629376 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bddf4ddb8-zhtcl_922beef2-8ec5-40ae-89c0-ce53ff159809/kube-auth-proxy/0.log" Jul 05 00:55:15.928732 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:15.928704 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-q5zsp_99d67b88-480f-48fd-9e06-07d9b217d717/payload-pre-processing/0.log" Jul 05 00:55:16.057584 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:16.057547 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-mmxgl_025bd7ca-0892-4e06-a7c7-764a46975c78/payload-processing/0.log" Jul 05 00:55:16.506616 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:16.506569 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-cvglp_ac6934f5-58c6-45ea-9a7b-7a7b7023bda1/main/0.log" Jul 05 00:55:16.626222 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:16.626182 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-m9k4g_a4951f64-509d-4d21-827c-602b6d8d904d/main/0.log" Jul 05 00:55:16.849341 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:16.849248 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt8k78_bc02abe7-2fe7-4d2f-9158-0ced489b3a00/main/0.log" Jul 05 00:55:21.803178 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:21.803149 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:55:21.901883 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:21.901847 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" limit=1024 Jul 05 00:55:24.619575 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:24.619539 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qb4z2_7ef46745-1efb-44fe-8511-59c406acd812/global-pull-secret-syncer/0.log" Jul 05 00:55:24.664483 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:24.664461 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qlbnt_fa9b29e8-905f-4113-9dd6-cde4021610d8/konnectivity-agent/0.log" Jul 05 00:55:24.737387 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:24.737353 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-0.ec2.internal_fb4e5006e9150c09aadf1dc6b5f68292/haproxy/0.log" Jul 05 00:55:29.979810 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:29.979777 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-cv5tv_93a20e30-fa1f-4676-bbfe-ec0005381944/manager/0.log" Jul 05 00:55:32.097046 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:32.097016 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_99a275a4-2248-4071-90dd-5c36deab90f6/alertmanager/0.log" Jul 05 00:55:32.116040 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:32.116013 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_99a275a4-2248-4071-90dd-5c36deab90f6/config-reloader/0.log" Jul 05 00:55:32.131027 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:32.131005 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_99a275a4-2248-4071-90dd-5c36deab90f6/kube-rbac-proxy-web/0.log" Jul 05 00:55:32.146126 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:32.146109 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_99a275a4-2248-4071-90dd-5c36deab90f6/kube-rbac-proxy/0.log" Jul 05 00:55:32.161427 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:32.161410 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_99a275a4-2248-4071-90dd-5c36deab90f6/kube-rbac-proxy-metric/0.log" Jul 05 00:55:32.180156 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:32.180133 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_99a275a4-2248-4071-90dd-5c36deab90f6/prom-label-proxy/0.log" Jul 05 00:55:32.197473 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:32.197457 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_99a275a4-2248-4071-90dd-5c36deab90f6/init-config-reloader/0.log" Jul 05 00:55:32.243349 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:32.243320 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tbvtg_10b6eca4-7fae-43e3-aeec-52934b2be942/cluster-monitoring-operator/0.log" Jul 05 00:55:32.368799 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:32.368723 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5db2k_bebc5085-24b5-4c18-bef7-aabcb2944337/node-exporter/0.log" Jul 05 00:55:32.384285 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:32.384261 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5db2k_bebc5085-24b5-4c18-bef7-aabcb2944337/kube-rbac-proxy/0.log" Jul 05 00:55:32.399626 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:32.399606 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5db2k_bebc5085-24b5-4c18-bef7-aabcb2944337/init-textfile/0.log" Jul 05 00:55:32.787758 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:32.787735 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5795879b4-lwbd7_edc11376-16e8-4b39-a8ac-ab9da1ab56a4/telemeter-client/0.log" Jul 05 00:55:32.802816 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:32.802795 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5795879b4-lwbd7_edc11376-16e8-4b39-a8ac-ab9da1ab56a4/reload/0.log" Jul 05 00:55:32.818131 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:32.818108 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5795879b4-lwbd7_edc11376-16e8-4b39-a8ac-ab9da1ab56a4/kube-rbac-proxy/0.log" Jul 05 00:55:34.903474 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.903441 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n"] Jul 05 00:55:34.905781 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.903819 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e41886d9-e354-4364-9212-044fff3c9b61" containerName="maas-api" Jul 05 00:55:34.905781 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.903829 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e41886d9-e354-4364-9212-044fff3c9b61" containerName="maas-api" Jul 05 00:55:34.905781 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.903853 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a124ae1b-a5b0-4e08-a82c-3a91eccf5864" containerName="maas-api" Jul 05 00:55:34.905781 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.903858 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a124ae1b-a5b0-4e08-a82c-3a91eccf5864" containerName="maas-api" Jul 05 00:55:34.905781 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.903947 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e41886d9-e354-4364-9212-044fff3c9b61" containerName="maas-api" Jul 05 00:55:34.905781 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.903955 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a124ae1b-a5b0-4e08-a82c-3a91eccf5864" containerName="maas-api" Jul 05 00:55:34.906936 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.906921 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:34.909536 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.909507 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6nfx9\"/\"openshift-service-ca.crt\"" Jul 05 00:55:34.909653 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.909550 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6nfx9\"/\"kube-root-ca.crt\"" Jul 05 00:55:34.909653 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.909596 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6nfx9\"/\"default-dockercfg-jzs6w\"" Jul 05 00:55:34.916205 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.916181 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n"] Jul 05 00:55:34.951349 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.951323 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c38f241f-5590-46ec-8c1a-aae795dfe2da-sys\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:34.951349 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.951349 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c38f241f-5590-46ec-8c1a-aae795dfe2da-proc\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:34.951525 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.951366 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c38f241f-5590-46ec-8c1a-aae795dfe2da-lib-modules\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:34.951525 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.951384 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c38f241f-5590-46ec-8c1a-aae795dfe2da-podres\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:34.951525 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:34.951421 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xplqt\" (UniqueName: \"kubernetes.io/projected/c38f241f-5590-46ec-8c1a-aae795dfe2da-kube-api-access-xplqt\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:35.052149 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:35.052113 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c38f241f-5590-46ec-8c1a-aae795dfe2da-sys\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:35.052149 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:35.052147 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c38f241f-5590-46ec-8c1a-aae795dfe2da-proc\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:35.052336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:35.052168 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c38f241f-5590-46ec-8c1a-aae795dfe2da-lib-modules\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:35.052336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:35.052185 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c38f241f-5590-46ec-8c1a-aae795dfe2da-podres\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:35.052336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:35.052210 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xplqt\" (UniqueName: \"kubernetes.io/projected/c38f241f-5590-46ec-8c1a-aae795dfe2da-kube-api-access-xplqt\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:35.052336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:35.052236 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c38f241f-5590-46ec-8c1a-aae795dfe2da-sys\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:35.052336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:35.052247 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c38f241f-5590-46ec-8c1a-aae795dfe2da-proc\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:35.052336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:35.052314 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c38f241f-5590-46ec-8c1a-aae795dfe2da-lib-modules\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:35.052336 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:35.052321 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c38f241f-5590-46ec-8c1a-aae795dfe2da-podres\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:35.059845 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:35.059816 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xplqt\" (UniqueName: \"kubernetes.io/projected/c38f241f-5590-46ec-8c1a-aae795dfe2da-kube-api-access-xplqt\") pod \"perf-node-gather-daemonset-rvw5n\" (UID: \"c38f241f-5590-46ec-8c1a-aae795dfe2da\") " pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:35.213274 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:35.213184 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-49j5s_a8354f6b-5bb0-4e2d-ac7e-60c022ebca48/download-server/0.log" Jul 05 00:55:35.217497 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:35.217471 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:35.349888 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:35.349864 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n"] Jul 05 00:55:35.352039 ip-10-0-130-0 kubenswrapper[2558]: W0705 00:55:35.352011 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc38f241f_5590_46ec_8c1a_aae795dfe2da.slice/crio-5b14651094110238c673750809d0985e7da248c200e59c9c8ad929f7e8154db6 WatchSource:0}: Error finding container 5b14651094110238c673750809d0985e7da248c200e59c9c8ad929f7e8154db6: Status 404 returned error can't find the container with id 5b14651094110238c673750809d0985e7da248c200e59c9c8ad929f7e8154db6 Jul 05 00:55:36.173698 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:36.173663 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" event={"ID":"c38f241f-5590-46ec-8c1a-aae795dfe2da","Type":"ContainerStarted","Data":"c78b2f24e26733f3f3833ac294b900e38f2919b16a511a8ddf4ccc6ea9b186ce"} Jul 05 00:55:36.174096 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:36.173723 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" event={"ID":"c38f241f-5590-46ec-8c1a-aae795dfe2da","Type":"ContainerStarted","Data":"5b14651094110238c673750809d0985e7da248c200e59c9c8ad929f7e8154db6"} Jul 05 00:55:36.174096 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:36.173776 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:36.189666 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:36.189612 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" podStartSLOduration=2.189596435 podStartE2EDuration="2.189596435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 00:55:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 00:55:36.189126481 +0000 UTC m=+2774.125491860" watchObservedRunningTime="2026-07-05 00:55:36.189596435 +0000 UTC m=+2774.125961832" Jul 05 00:55:36.703606 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:36.703568 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-s764k_dd7a485d-b0d9-4e77-bf59-bdf7ed11f365/dns/0.log" Jul 05 00:55:36.719642 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:36.719617 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-s764k_dd7a485d-b0d9-4e77-bf59-bdf7ed11f365/kube-rbac-proxy/0.log" Jul 05 00:55:36.770397 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:36.770366 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9595p_215cde65-c249-45c8-b941-488c4ab6b568/dns-node-resolver/0.log" Jul 05 00:55:37.297320 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:37.297283 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5cb9dfb49c-cdph4_41415125-9abb-4915-832f-7b8dbf7fd381/registry/0.log" Jul 05 00:55:37.316498 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:37.316472 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8s5zw_f2fde0b3-a812-4baf-9da4-719e7b227fd9/node-ca/0.log" Jul 05 00:55:38.365240 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:38.365210 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fsbtc_50249fa7-6685-4f7d-9ba2-feed1eedcbf0/discovery/0.log" Jul 05 00:55:38.399544 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:38.399509 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bddf4ddb8-zhtcl_922beef2-8ec5-40ae-89c0-ce53ff159809/kube-auth-proxy/0.log" Jul 05 00:55:38.490498 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:38.490464 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-q5zsp_99d67b88-480f-48fd-9e06-07d9b217d717/payload-pre-processing/0.log" Jul 05 00:55:38.560630 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:38.560600 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-mmxgl_025bd7ca-0892-4e06-a7c7-764a46975c78/payload-processing/0.log" Jul 05 00:55:39.198547 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:39.198507 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c9fwj_a9df4b2b-5e36-47e0-a1eb-fe2bd1aa44a4/serve-healthcheck-canary/0.log" Jul 05 00:55:39.703514 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:39.703488 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lw7zc_37f1b5d2-c173-4744-a850-6053f1f3bd35/kube-rbac-proxy/0.log" Jul 05 00:55:39.721594 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:39.721568 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lw7zc_37f1b5d2-c173-4744-a850-6053f1f3bd35/exporter/0.log" Jul 05 00:55:39.737619 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:39.737598 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lw7zc_37f1b5d2-c173-4744-a850-6053f1f3bd35/extractor/0.log" Jul 05 00:55:41.816667 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:41.816640 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-79bvr_a25ed367-e43c-4191-86d6-557cb7f14e4a/manager/0.log" Jul 05 00:55:42.047362 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:42.047329 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vgkg6_3e9b555f-2665-44e7-95b2-9587f4863760/manager/0.log" Jul 05 00:55:42.094562 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:42.094490 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5b96d4fbd4-5b8hr_c1f82e5b-362d-485e-980e-23891749ff79/maas-api/0.log" Jul 05 00:55:42.190436 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:42.190406 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6nfx9/perf-node-gather-daemonset-rvw5n" Jul 05 00:55:42.293334 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:42.293293 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-98468dc4b-db98h_ae82ae7b-1baf-4c76-b71f-5f3efef3df37/manager/0.log" Jul 05 00:55:42.309386 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:42.309359 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bm469_10e57536-ef8d-4e8c-a817-907de7fe3366/server/0.log" Jul 05 00:55:42.421676 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:42.421642 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dzcz4_28b5b72c-0953-4c98-a9a1-d45eab3be5d3/manager/0.log" Jul 05 00:55:42.443814 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:42.443789 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674c4bd948-84wl8_8a6ad246-4f69-4fbf-9cd2-fed27a05502e/manager/0.log" Jul 05 00:55:42.460610 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:42.460587 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674c4bd948-84wl8_8a6ad246-4f69-4fbf-9cd2-fed27a05502e/copy-manifests/0.log" Jul 05 00:55:42.611496 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:42.611413 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-8cpxr_6da95d6e-6b88-4f8b-9472-f44ab80723cc/postgres/0.log" Jul 05 00:55:44.050855 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:44.050773 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-ml8r4_6c7f13da-2f23-492b-a34d-970e1ec64026/openshift-lws-operator/0.log" Jul 05 00:55:44.984153 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:44.984111 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fsbtc_50249fa7-6685-4f7d-9ba2-feed1eedcbf0/discovery/0.log" Jul 05 00:55:45.005753 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:45.005729 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bddf4ddb8-zhtcl_922beef2-8ec5-40ae-89c0-ce53ff159809/kube-auth-proxy/0.log" Jul 05 00:55:45.086719 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:45.086682 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-q5zsp_99d67b88-480f-48fd-9e06-07d9b217d717/payload-pre-processing/0.log" Jul 05 00:55:45.146506 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:45.146474 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-mmxgl_025bd7ca-0892-4e06-a7c7-764a46975c78/payload-processing/0.log" Jul 05 00:55:46.523070 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:46.523035 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fsbtc_50249fa7-6685-4f7d-9ba2-feed1eedcbf0/discovery/0.log" Jul 05 00:55:46.546595 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:46.546569 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bddf4ddb8-zhtcl_922beef2-8ec5-40ae-89c0-ce53ff159809/kube-auth-proxy/0.log" Jul 05 00:55:46.617893 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:46.617797 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-q5zsp_99d67b88-480f-48fd-9e06-07d9b217d717/payload-pre-processing/0.log" Jul 05 00:55:46.682205 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:46.682173 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-mmxgl_025bd7ca-0892-4e06-a7c7-764a46975c78/payload-processing/0.log" Jul 05 00:55:50.393087 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:50.393050 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8wn9_039c4612-e8d2-4bf8-994f-37600257cc9b/kube-multus-additional-cni-plugins/0.log" Jul 05 00:55:50.408468 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:50.408445 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8wn9_039c4612-e8d2-4bf8-994f-37600257cc9b/egress-router-binary-copy/0.log" Jul 05 00:55:50.423948 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:50.423926 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8wn9_039c4612-e8d2-4bf8-994f-37600257cc9b/cni-plugins/0.log" Jul 05 00:55:50.438769 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:50.438740 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8wn9_039c4612-e8d2-4bf8-994f-37600257cc9b/bond-cni-plugin/0.log" Jul 05 00:55:50.453395 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:50.453376 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8wn9_039c4612-e8d2-4bf8-994f-37600257cc9b/routeoverride-cni/0.log" Jul 05 00:55:50.470541 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:50.470521 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8wn9_039c4612-e8d2-4bf8-994f-37600257cc9b/whereabouts-cni-bincopy/0.log" Jul 05 00:55:50.488520 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:50.488494 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8wn9_039c4612-e8d2-4bf8-994f-37600257cc9b/whereabouts-cni/0.log" Jul 05 00:55:50.548127 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:50.548101 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mxnwd_eda5c663-9ac6-4cbf-a3bc-eb37fbe649d1/kube-multus/0.log" Jul 05 00:55:50.643620 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:50.643584 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-td7ll_a8e8f787-142b-4f26-9a75-9cc3b6976f35/network-metrics-daemon/0.log" Jul 05 00:55:50.658039 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:50.658012 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-td7ll_a8e8f787-142b-4f26-9a75-9cc3b6976f35/kube-rbac-proxy/0.log" Jul 05 00:55:51.422878 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:51.422845 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56fqd_a828f18f-4034-4e6f-963f-04e391abbe14/ovn-controller/0.log" Jul 05 00:55:51.433848 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:51.433827 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56fqd_a828f18f-4034-4e6f-963f-04e391abbe14/ovn-acl-logging/0.log" Jul 05 00:55:51.457375 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:51.457348 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56fqd_a828f18f-4034-4e6f-963f-04e391abbe14/kube-rbac-proxy-node/0.log" Jul 05 00:55:51.476948 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:51.476924 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56fqd_a828f18f-4034-4e6f-963f-04e391abbe14/kube-rbac-proxy-ovn-metrics/0.log" Jul 05 00:55:51.489728 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:51.489707 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56fqd_a828f18f-4034-4e6f-963f-04e391abbe14/northd/0.log" Jul 05 00:55:51.504126 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:51.504106 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56fqd_a828f18f-4034-4e6f-963f-04e391abbe14/nbdb/0.log" Jul 05 00:55:51.519925 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:51.519890 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56fqd_a828f18f-4034-4e6f-963f-04e391abbe14/sbdb/0.log" Jul 05 00:55:51.588308 ip-10-0-130-0 kubenswrapper[2558]: I0705 00:55:51.588277 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56fqd_a828f18f-4034-4e6f-963f-04e391abbe14/ovnkube-controller/0.log"