Jul 06 21:19:21.771883 ip-10-0-131-78 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 21:19:21.771896 ip-10-0-131-78 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 21:19:21.771906 ip-10-0-131-78 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 21:19:21.772345 ip-10-0-131-78 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 21:19:31.790633 ip-10-0-131-78 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 21:19:31.790649 ip-10-0-131-78 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot b4778979b60c447ca6af37f8ed0205a8 -- Jul 06 21:21:31.022254 ip-10-0-131-78 systemd[1]: Starting Kubernetes Kubelet... Jul 06 21:21:33.802502 ip-10-0-131-78 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 21:21:33.802502 ip-10-0-131-78 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 21:21:33.802502 ip-10-0-131-78 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 21:21:33.802502 ip-10-0-131-78 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 21:21:33.802502 ip-10-0-131-78 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 21:21:33.802502 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.784288 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 21:21:33.802502 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788582 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 21:21:33.802502 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788594 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 21:21:33.802502 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788597 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 21:21:33.802502 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788600 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 21:21:33.802502 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788603 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 21:21:33.802502 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788606 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 21:21:33.802502 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788609 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 21:21:33.802502 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788611 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788614 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788617 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788619 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788622 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788624 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788628 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788631 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788633 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788636 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788638 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788640 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788643 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788645 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788648 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788650 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788653 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788655 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788659 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788662 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 21:21:33.804676 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788664 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788667 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788669 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788672 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788675 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788678 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788680 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788682 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788685 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788688 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788692 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788696 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788698 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788701 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788703 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788706 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788708 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788711 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788714 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788716 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 21:21:33.805198 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788720 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788722 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788725 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788727 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788730 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788732 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788735 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788737 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788740 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788742 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788745 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788748 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788751 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788753 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788756 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788759 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788762 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788764 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788767 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788770 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 21:21:33.805931 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788772 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788776 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788780 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788784 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788787 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788790 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788793 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788795 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788797 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788800 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788804 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788807 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788809 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788812 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788814 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788816 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788819 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788821 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788823 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 21:21:33.806457 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788826 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788829 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788831 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788834 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788836 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788839 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788842 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788844 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788847 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788850 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788852 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788855 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.788857 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789551 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789558 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789561 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789564 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789567 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789569 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789573 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 21:21:33.931964 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789575 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 21:21:33.872090 ip-10-0-131-78 systemd[1]: Started Kubernetes Kubelet. Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789578 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789581 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789583 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789586 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789589 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789591 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789595 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789599 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789601 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789604 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789607 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789610 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789612 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789615 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789617 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789620 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789622 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789625 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789627 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 21:21:33.933956 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789631 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789633 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789636 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789639 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789641 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789644 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789646 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789649 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789651 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789654 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789657 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789659 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789662 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789664 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789667 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789672 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789674 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789677 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789679 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 21:21:33.934760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789682 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789685 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789689 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789692 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789695 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789698 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789700 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789704 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789707 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789709 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789712 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789714 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789717 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789719 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789722 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789724 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789727 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789729 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789732 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 21:21:33.936771 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789734 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789737 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789739 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789742 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789745 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789747 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789750 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789752 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789755 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789762 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789764 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789767 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789769 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789773 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789776 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789779 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789781 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789784 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789788 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789791 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 21:21:33.937583 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789794 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789797 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789800 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789803 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789805 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789808 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789811 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789814 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789817 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789819 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789822 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789825 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789828 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.789830 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791816 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791830 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791833 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791836 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791839 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791842 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 21:21:33.938795 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791845 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791848 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791852 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791855 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791857 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791860 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791863 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791866 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791868 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791871 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791873 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791876 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791878 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791881 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791884 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791886 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791890 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791892 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791895 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791898 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 21:21:33.939610 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791900 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791902 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791910 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791913 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791916 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791918 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791921 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791926 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791929 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791932 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791934 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791937 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791939 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791960 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791963 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791966 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791968 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791971 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791974 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791976 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 21:21:33.940801 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791979 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791981 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791984 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791986 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791989 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791991 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791994 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791996 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.791999 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792001 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792004 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792006 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792009 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792011 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792014 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792019 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792022 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792025 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792027 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 21:21:33.942825 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792033 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792038 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792041 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792043 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792049 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792054 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792057 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792060 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792062 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792065 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792068 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792070 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792073 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792075 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792078 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792081 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792083 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792086 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792089 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792092 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 21:21:33.943360 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792094 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792097 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792099 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792102 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792105 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792107 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792109 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792112 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792114 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792117 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792120 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792123 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792129 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792132 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792198 2573 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792205 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792212 2573 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792217 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792221 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792224 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792229 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 21:21:33.943832 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792233 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792236 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792239 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792243 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792246 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792249 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792252 2573 flags.go:64] FLAG: --cgroup-root="" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792255 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792258 2573 flags.go:64] FLAG: --client-ca-file="" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792261 2573 flags.go:64] FLAG: --cloud-provider="external" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792265 2573 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792270 2573 flags.go:64] FLAG: --cluster-domain="" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792272 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792275 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792278 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792281 2573 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792285 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792288 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792291 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792294 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792297 2573 flags.go:64] FLAG: --contention-profiling="false" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792300 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792303 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792308 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 21:21:33.944349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792311 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792315 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792318 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792321 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792324 2573 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792327 2573 flags.go:64] FLAG: --enable-server="true" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792330 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792334 2573 flags.go:64] FLAG: --event-burst="100" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792337 2573 flags.go:64] FLAG: --event-qps="50" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792340 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792343 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792346 2573 flags.go:64] FLAG: --eviction-hard="" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792349 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792352 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792355 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792358 2573 flags.go:64] FLAG: --eviction-soft="" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792361 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792363 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792366 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792370 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792373 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792376 2573 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792379 2573 flags.go:64] FLAG: --feature-gates="" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792382 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792385 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 21:21:33.944895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792388 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792391 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792394 2573 flags.go:64] FLAG: --healthz-port="10248" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792397 2573 flags.go:64] FLAG: --help="false" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792400 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-131-78.ec2.internal" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792403 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792408 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792411 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792415 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792418 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792421 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792423 2573 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792426 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792429 2573 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792432 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792435 2573 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792438 2573 flags.go:64] FLAG: --kube-reserved="" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792440 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792443 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792446 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792448 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792451 2573 flags.go:64] FLAG: --lock-file="" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792454 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792457 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 21:21:33.945548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792459 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792465 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792467 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792471 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792474 2573 flags.go:64] FLAG: --logging-format="text" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792476 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792479 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792482 2573 flags.go:64] FLAG: --manifest-url="" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792485 2573 flags.go:64] FLAG: --manifest-url-header="" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792489 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792492 2573 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792495 2573 flags.go:64] FLAG: --max-pods="110" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792498 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792501 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792507 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792510 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792513 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792516 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792519 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792527 2573 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792530 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792533 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792536 2573 flags.go:64] FLAG: --pod-cidr="" Jul 06 21:21:33.946127 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792538 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792544 2573 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792547 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792550 2573 flags.go:64] FLAG: --pods-per-core="0" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792553 2573 flags.go:64] FLAG: --port="10250" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792556 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792558 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06acdafe0316e8a47" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792561 2573 flags.go:64] FLAG: --qos-reserved="" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792564 2573 flags.go:64] FLAG: --read-only-port="10255" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792567 2573 flags.go:64] FLAG: --register-node="true" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792570 2573 flags.go:64] FLAG: --register-with-taints="" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792578 2573 flags.go:64] FLAG: --registry-burst="10" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792581 2573 flags.go:64] FLAG: --registry-qps="5" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792584 2573 flags.go:64] FLAG: --reserved-cpus="" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792587 2573 flags.go:64] FLAG: --reserved-memory="" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792590 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792593 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792596 2573 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792599 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792602 2573 flags.go:64] FLAG: --runonce="false" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792605 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792608 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792611 2573 flags.go:64] FLAG: --seccomp-default="false" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792616 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792620 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792623 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 21:21:33.946674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792626 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792629 2573 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792632 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792634 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792637 2573 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792640 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792643 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792646 2573 flags.go:64] FLAG: --system-cgroups="" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792648 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792654 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792657 2573 flags.go:64] FLAG: --tls-cert-file="" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792659 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792664 2573 flags.go:64] FLAG: --tls-min-version="" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792667 2573 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792669 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792672 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792675 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792678 2573 flags.go:64] FLAG: --v="2" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792681 2573 flags.go:64] FLAG: --version="false" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792685 2573 flags.go:64] FLAG: --vmodule="" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792689 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.792692 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792802 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792806 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 21:21:33.947304 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792809 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792812 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792815 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792817 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792820 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792824 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792828 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792831 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792833 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792836 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792840 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792843 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792847 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792849 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792852 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792855 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792858 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792860 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792863 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 21:21:33.947864 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792865 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792868 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792870 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792873 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792875 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792878 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792881 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792883 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792886 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792888 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792891 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792894 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792896 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792899 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792901 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792903 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792906 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792908 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792912 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792916 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 21:21:33.948326 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792919 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792922 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792924 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792927 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792929 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792932 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792934 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792937 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792963 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792968 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792970 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792973 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792976 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792979 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792981 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792984 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792986 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792990 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792994 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 21:21:33.948820 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792996 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.792999 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793003 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793006 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793009 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793011 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793014 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793017 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793019 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793022 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793025 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793029 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793033 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793035 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793039 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793041 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793044 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793046 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793049 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793051 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 21:21:33.949288 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793054 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793056 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793059 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793062 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793064 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793067 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793069 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793072 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793074 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793077 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793079 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793082 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793085 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793088 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793091 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793094 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793096 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793100 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 21:21:33.949760 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.793103 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 21:21:33.950200 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.793138 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 21:21:33.950200 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.799912 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 21:21:33.950200 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.799928 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 21:21:33.950200 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.799959 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 21:21:33.950200 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.799968 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 21:21:33.950200 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800020 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 21:21:33.950200 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800025 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 21:21:33.950200 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800030 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 21:21:33.950200 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800033 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 21:21:33.950200 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800036 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 21:21:33.950200 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800039 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 21:21:33.950200 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800041 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 21:21:33.950200 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800044 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 21:21:33.950200 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800047 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800050 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800053 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800055 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800058 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800060 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800063 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800066 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800068 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800071 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800073 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800076 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800079 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800081 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800084 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800087 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800089 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800094 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800097 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 21:21:33.950544 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800104 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800107 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800111 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800114 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800117 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800120 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800123 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800126 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800129 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800132 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800134 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800137 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800140 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800142 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800145 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800147 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800150 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800153 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800155 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800158 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 21:21:33.950992 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800160 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800163 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800165 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800168 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800171 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800173 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800176 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800178 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800181 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800183 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800186 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800190 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800193 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800196 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800203 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800206 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800208 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800211 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800213 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 21:21:33.951469 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800216 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800219 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800221 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800224 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800227 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800229 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800232 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800234 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800237 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800240 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800242 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800245 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800248 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800250 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800253 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800256 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800258 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800261 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800263 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800266 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 21:21:33.951911 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800268 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800271 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800274 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800276 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800279 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800285 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800288 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800293 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800295 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800298 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800300 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800304 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800308 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.800313 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800442 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 21:21:33.952390 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800446 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800450 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800453 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800455 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800458 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800461 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800464 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800467 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800470 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800472 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800475 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800478 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800480 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800483 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800485 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800488 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800490 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800493 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800495 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800498 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 21:21:33.952753 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800501 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800503 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800506 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800511 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800515 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800517 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800520 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800522 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800525 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800527 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800530 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800532 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800535 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800537 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800540 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800542 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800545 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800547 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800550 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800553 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 21:21:33.953256 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800555 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800558 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800560 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800563 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800566 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800568 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800570 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800573 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800575 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800578 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800580 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800583 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800585 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800587 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800590 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800594 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800600 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800602 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800605 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800608 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 21:21:33.953720 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800610 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800613 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800616 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800620 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800623 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800626 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800629 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800632 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800635 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800637 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800640 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800643 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800647 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800650 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800653 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800656 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800659 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800661 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800664 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 21:21:33.954232 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800667 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800669 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800672 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800674 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800677 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800679 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800682 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800686 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800690 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800695 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800698 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800701 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800704 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800707 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800709 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800712 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800714 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800717 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 21:21:33.954681 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:33.800719 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 21:21:33.955130 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.800724 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 21:21:33.955130 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.800817 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 21:21:33.955130 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.802742 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 21:21:33.955130 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.804936 2573 server.go:1019] "Starting client certificate rotation" Jul 06 21:21:33.955130 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.805057 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 21:21:33.955130 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.805094 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 21:21:33.955130 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.807355 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 21:21:33.955130 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.809311 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 21:21:33.955130 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.821445 2573 log.go:25] "Validated CRI v1 runtime API" Jul 06 21:21:33.955130 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.823590 2573 log.go:25] "Validated CRI v1 image API" Jul 06 21:21:33.955130 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.824399 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 21:21:33.955130 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.828522 2573 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 811adc6a-423e-44fc-9c58-52b738a57bd4:/dev/nvme0n1p4 c7bd7ac6-3724-486b-8a6b-704ce72156d8:/dev/nvme0n1p3] Jul 06 21:21:33.955423 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.828586 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 21:21:33.955457 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.833867 2573 manager.go:216] Machine: {Timestamp:2026-07-06 21:21:33.833401114 +0000 UTC m=+1.585024169 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100325 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22434c06709e4a03f5aa592905c123 SystemUUID:ec22434c-0670-9e4a-03f5-aa592905c123 BootID:b4778979-b60c-447c-a6af-37f8ed0205a8 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2d:f5:75:d6:cd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2d:f5:75:d6:cd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:2b:0d:21:83:37 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 21:21:33.955457 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.834006 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 21:21:33.955457 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.834081 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 21:21:33.955457 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.838382 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 21:21:33.955457 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.838750 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 21:21:33.955457 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.838771 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-78.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 21:21:33.955457 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.840386 2573 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.840396 2573 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.840404 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.841155 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.842776 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.847709 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.847817 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.847844 2573 kubelet.go:486] "Attempting to sync node with API server" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.847853 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.847863 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.847871 2573 kubelet.go:398] "Adding apiserver pod source" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.847892 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.848216 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.848240 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.856750 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.858979 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.859007 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.860189 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.860202 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.860208 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.860213 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.860222 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.860227 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.860235 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.860240 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.860246 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.860251 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.860259 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.860264 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.861665 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.861672 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.864607 2573 server.go:1268] "Started kubelet" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.864663 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.864697 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.864778 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.866983 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.872379 2573 server.go:310] "Adding debug handlers to kubelet server" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.877540 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.877548 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:33.877757 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-78.ec2.internal\" not found" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:33.877852 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-78.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 21:21:33.955663 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:33.877850 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 21:21:33.956648 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.878380 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-78.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 21:21:33.956648 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.878460 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 21:21:33.956648 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.878461 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 21:21:33.956648 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.878479 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 21:21:33.956648 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:33.877315 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-78.ec2.internal.18bfd032c0dbd507 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-78.ec2.internal,UID:ip-10-0-131-78.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-78.ec2.internal,},FirstTimestamp:2026-07-06 21:21:33.864580359 +0000 UTC m=+1.616203414,LastTimestamp:2026-07-06 21:21:33.864580359 +0000 UTC m=+1.616203414,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-78.ec2.internal,}" Jul 06 21:21:33.956648 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.878577 2573 reconstruct.go:97] "Volume reconstruction finished" Jul 06 21:21:33.956648 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.878585 2573 reconciler.go:29] "Reconciler: start to sync state" Jul 06 21:21:33.956648 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:33.884685 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 21:21:33.956648 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:33.884746 2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-78.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 21:21:33.956648 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.893113 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rv68s" Jul 06 21:21:33.956648 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:33.906824 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rv68s" Jul 06 21:21:33.978766 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:33.978738 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-78.ec2.internal\" not found" Jul 06 21:21:34.043069 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.043041 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 21:21:34.044892 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.044875 2573 factory.go:55] Registering systemd factory Jul 06 21:21:34.044892 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.044893 2573 factory.go:223] Registration of the systemd container factory successfully Jul 06 21:21:34.045138 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.045125 2573 factory.go:153] Registering CRI-O factory Jul 06 21:21:34.045175 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.045140 2573 factory.go:223] Registration of the crio container factory successfully Jul 06 21:21:34.045215 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.045205 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 21:21:34.045250 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.045233 2573 factory.go:103] Registering Raw factory Jul 06 21:21:34.045250 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.045246 2573 manager.go:1196] Started watching for new ooms in manager Jul 06 21:21:34.045878 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.045866 2573 manager.go:349] Starting recovery of all containers Jul 06 21:21:34.055180 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.055139 2573 manager.go:354] Recovery completed Jul 06 21:21:34.060189 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.060175 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 21:21:34.062436 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.062417 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasSufficientMemory" Jul 06 21:21:34.062572 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.062554 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 21:21:34.062872 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.062855 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasSufficientPID" Jul 06 21:21:34.063483 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.063463 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 21:21:34.063483 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.063477 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 21:21:34.063618 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.063478 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 21:21:34.063618 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.063562 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 06 21:21:34.064266 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.064251 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasSufficientMemory" Jul 06 21:21:34.064336 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.064281 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 21:21:34.064336 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.064295 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasSufficientPID" Jul 06 21:21:34.065156 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.065039 2573 policy_none.go:49] "None policy: Start" Jul 06 21:21:34.065156 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.065065 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 21:21:34.065156 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.065079 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 21:21:34.065156 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.065100 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-78.ec2.internal.18bfd032cca84793 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-78.ec2.internal,UID:ip-10-0-131-78.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-78.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-78.ec2.internal,},FirstTimestamp:2026-07-06 21:21:34.062528403 +0000 UTC m=+1.814151460,LastTimestamp:2026-07-06 21:21:34.062528403 +0000 UTC m=+1.814151460,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-78.ec2.internal,}" Jul 06 21:21:34.066302 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.066291 2573 policy_none.go:47] "Start" Jul 06 21:21:34.079619 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.079595 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-78.ec2.internal\" not found" Jul 06 21:21:34.099645 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.099626 2573 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-78.ec2.internal\" not found" node="ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.101491 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.101479 2573 manager.go:337] "Starting Device Plugin manager" Jul 06 21:21:34.101530 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.101520 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 21:21:34.101530 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.101529 2573 server.go:87] "Starting device plugin registration server" Jul 06 21:21:34.101810 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.101797 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 21:21:34.101863 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.101812 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 21:21:34.101930 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.101913 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 21:21:34.102028 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.102012 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 21:21:34.102028 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.102025 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 21:21:34.102574 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.102555 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 21:21:34.102642 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.102606 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-78.ec2.internal\" not found" Jul 06 21:21:34.201973 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.201933 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 21:21:34.203989 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.202864 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasSufficientMemory" Jul 06 21:21:34.203989 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.202893 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 21:21:34.203989 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.202922 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasSufficientPID" Jul 06 21:21:34.203989 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.202967 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.210064 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.210041 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 21:21:34.211258 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.211236 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 21:21:34.211372 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.211272 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 21:21:34.211372 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.211291 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 21:21:34.211372 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.211320 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 21:21:34.213321 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.213305 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.213411 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.213329 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-78.ec2.internal\": node \"ip-10-0-131-78.ec2.internal\" not found" Jul 06 21:21:34.216983 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.216967 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 21:21:34.261790 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.261763 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-78.ec2.internal\" not found" Jul 06 21:21:34.312332 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.312255 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-78.ec2.internal"] Jul 06 21:21:34.312435 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.312344 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 21:21:34.314029 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.314014 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasSufficientMemory" Jul 06 21:21:34.314099 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.314044 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 21:21:34.314099 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.314053 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasSufficientPID" Jul 06 21:21:34.315321 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.315308 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 21:21:34.315443 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.315429 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.315515 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.315476 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 21:21:34.317005 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.316984 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasSufficientMemory" Jul 06 21:21:34.317105 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.317013 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 21:21:34.317105 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.317026 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasSufficientPID" Jul 06 21:21:34.317105 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.316988 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasSufficientMemory" Jul 06 21:21:34.317105 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.317077 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 21:21:34.317105 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.317099 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasSufficientPID" Jul 06 21:21:34.318326 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.318309 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.318413 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.318333 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 21:21:34.319582 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.319570 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasSufficientMemory" Jul 06 21:21:34.319646 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.319596 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 21:21:34.319646 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.319607 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeHasSufficientPID" Jul 06 21:21:34.342441 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.342424 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-78.ec2.internal\" not found" node="ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.346495 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.346479 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-78.ec2.internal\" not found" node="ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.362607 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.362589 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-78.ec2.internal\" not found" Jul 06 21:21:34.380207 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.380188 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91fb7639492fa2b32fb2be643b9b7d5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal\" (UID: \"91fb7639492fa2b32fb2be643b9b7d5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.380260 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.380214 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91fb7639492fa2b32fb2be643b9b7d5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal\" (UID: \"91fb7639492fa2b32fb2be643b9b7d5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.380260 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.380231 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02bd88d5d5006b9faa87b2a2edfa6ac4-config\") pod \"kube-apiserver-proxy-ip-10-0-131-78.ec2.internal\" (UID: \"02bd88d5d5006b9faa87b2a2edfa6ac4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.463358 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.463329 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-78.ec2.internal\" not found" Jul 06 21:21:34.480412 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.480391 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02bd88d5d5006b9faa87b2a2edfa6ac4-config\") pod \"kube-apiserver-proxy-ip-10-0-131-78.ec2.internal\" (UID: \"02bd88d5d5006b9faa87b2a2edfa6ac4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.480468 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.480416 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91fb7639492fa2b32fb2be643b9b7d5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal\" (UID: \"91fb7639492fa2b32fb2be643b9b7d5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.480468 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.480434 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91fb7639492fa2b32fb2be643b9b7d5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal\" (UID: \"91fb7639492fa2b32fb2be643b9b7d5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.480536 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.480471 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91fb7639492fa2b32fb2be643b9b7d5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal\" (UID: \"91fb7639492fa2b32fb2be643b9b7d5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.480536 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.480503 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02bd88d5d5006b9faa87b2a2edfa6ac4-config\") pod \"kube-apiserver-proxy-ip-10-0-131-78.ec2.internal\" (UID: \"02bd88d5d5006b9faa87b2a2edfa6ac4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.480536 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.480505 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91fb7639492fa2b32fb2be643b9b7d5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal\" (UID: \"91fb7639492fa2b32fb2be643b9b7d5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.563710 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.563638 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-78.ec2.internal\" not found" Jul 06 21:21:34.648171 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.648133 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.650694 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.650673 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.663917 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.663893 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-78.ec2.internal\" not found" Jul 06 21:21:34.747574 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.747547 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 21:21:34.764579 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.764548 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-78.ec2.internal\" not found" Jul 06 21:21:34.804022 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.803989 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 21:21:34.804461 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.804129 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 21:21:34.804461 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.804165 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 21:21:34.864759 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:34.864701 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-78.ec2.internal\" not found" Jul 06 21:21:34.866890 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.866873 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 21:21:34.877936 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.877911 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 21:21:34.878075 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.877936 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.902649 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.902625 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 21:21:34.903590 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.903574 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 21:21:34.905226 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.905209 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-78.ec2.internal" Jul 06 21:21:34.908616 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.908594 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 21:16:33 +0000 UTC" deadline="2028-01-12 15:37:42.829342232 +0000 UTC" Jul 06 21:21:34.908616 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.908615 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13314h16m7.920729695s" Jul 06 21:21:34.920974 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.920940 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 21:21:34.964045 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.964019 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fqvtc" Jul 06 21:21:34.990759 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:34.990736 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fqvtc" Jul 06 21:21:35.225885 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:35.225855 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91fb7639492fa2b32fb2be643b9b7d5e.slice/crio-8902baebc44f683edffe9a8f086fe09442d7ffd98df62aadfa7b7c2459fe3d5c WatchSource:0}: Error finding container 8902baebc44f683edffe9a8f086fe09442d7ffd98df62aadfa7b7c2459fe3d5c: Status 404 returned error can't find the container with id 8902baebc44f683edffe9a8f086fe09442d7ffd98df62aadfa7b7c2459fe3d5c Jul 06 21:21:35.226548 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:35.226526 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02bd88d5d5006b9faa87b2a2edfa6ac4.slice/crio-05b5dcc10874af3203f9cbb775010ae29a3a90f5c88b0e9eecac213c0cb3fafd WatchSource:0}: Error finding container 05b5dcc10874af3203f9cbb775010ae29a3a90f5c88b0e9eecac213c0cb3fafd: Status 404 returned error can't find the container with id 05b5dcc10874af3203f9cbb775010ae29a3a90f5c88b0e9eecac213c0cb3fafd Jul 06 21:21:35.234471 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.234456 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 21:21:35.473866 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.473778 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 21:21:35.849178 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.848963 2573 apiserver.go:52] "Watching apiserver" Jul 06 21:21:35.856802 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.856775 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 21:21:35.857329 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.857280 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-dctfx","openshift-cluster-node-tuning-operator/tuned-wtb6f","openshift-dns/node-resolver-qmqnl","openshift-image-registry/node-ca-7djjr","openshift-multus/network-metrics-daemon-b8dk6","openshift-network-diagnostics/network-check-target-zx6nz","openshift-ovn-kubernetes/ovnkube-node-mfjvh","kube-system/global-pull-secret-syncer-zb6q8","kube-system/kube-apiserver-proxy-ip-10-0-131-78.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal","openshift-multus/multus-additional-cni-plugins-vsbrr","openshift-multus/multus-p2t6l","openshift-network-operator/iptables-alerter-9npmj"] Jul 06 21:21:35.858834 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.858803 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dctfx" Jul 06 21:21:35.860490 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.860474 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.861959 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.861924 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qmqnl" Jul 06 21:21:35.864983 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.863116 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 21:21:35.864983 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.863335 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-577qj\"" Jul 06 21:21:35.864983 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.863383 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5f297\"" Jul 06 21:21:35.864983 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.863522 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 21:21:35.864983 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.863638 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 21:21:35.864983 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.863653 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7djjr" Jul 06 21:21:35.864983 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.863754 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 21:21:35.865357 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.865109 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 21:21:35.865357 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.865323 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:35.865454 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:35.865381 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:21:35.867512 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.866332 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 21:21:35.867512 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.866924 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 21:21:35.867512 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.867251 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 21:21:35.867512 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.867353 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 21:21:35.867512 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.867385 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nl8nk\"" Jul 06 21:21:35.867512 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.867423 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cztsh\"" Jul 06 21:21:35.867836 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.867542 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:35.867836 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:35.867655 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:21:35.869478 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.869461 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.872472 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.872322 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 21:21:35.872472 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.872390 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:35.872472 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:35.872441 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:21:35.872666 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.872647 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5hv6b\"" Jul 06 21:21:35.872900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.872881 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 21:21:35.873667 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.873646 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 21:21:35.873754 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.873700 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 21:21:35.873832 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.873816 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 21:21:35.874020 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.874005 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 21:21:35.874087 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.874061 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.876341 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.876321 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.876937 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.876918 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 21:21:35.877052 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.877022 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 21:21:35.877118 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.877086 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sb888\"" Jul 06 21:21:35.877243 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.877226 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 21:21:35.880239 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.880220 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 21:21:35.880239 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.880229 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 21:21:35.880382 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.880261 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 21:21:35.883004 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.881404 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 21:21:35.883004 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.881886 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-thzln\"" Jul 06 21:21:35.883004 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.882011 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.883631 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.883282 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 21:21:35.887254 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.886145 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9b46j\"" Jul 06 21:21:35.887485 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.887431 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9npmj" Jul 06 21:21:35.888230 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888208 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2692f890-289e-4696-9a23-94c43e3a991c-agent-certs\") pod \"konnectivity-agent-dctfx\" (UID: \"2692f890-289e-4696-9a23-94c43e3a991c\") " pod="kube-system/konnectivity-agent-dctfx" Jul 06 21:21:35.888328 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888252 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-tuned\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.888328 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888276 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89c932c7-a11a-4fb9-b862-fa08214a6335-tmp\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.888328 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888306 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-cnibin\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.888328 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888336 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-run-netns\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.888546 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888365 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-device-dir\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.888546 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888397 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5b843783-33a8-4332-8d96-a21b5abc3a40-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.888546 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888427 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5b843783-33a8-4332-8d96-a21b5abc3a40-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.888546 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888460 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2692f890-289e-4696-9a23-94c43e3a991c-konnectivity-ca\") pod \"konnectivity-agent-dctfx\" (UID: \"2692f890-289e-4696-9a23-94c43e3a991c\") " pod="kube-system/konnectivity-agent-dctfx" Jul 06 21:21:35.888546 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888490 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-system-cni-dir\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.888546 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888520 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-var-lib-cni-bin\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.888823 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888548 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnmt2\" (UniqueName: \"kubernetes.io/projected/e7231950-58b8-40d6-ad8d-7a57df972c1f-kube-api-access-vnmt2\") pod \"node-resolver-qmqnl\" (UID: \"e7231950-58b8-40d6-ad8d-7a57df972c1f\") " pod="openshift-dns/node-resolver-qmqnl" Jul 06 21:21:35.888823 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888589 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-kubelet\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.888823 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888620 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.888823 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888651 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-kubelet-config\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:35.888823 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888681 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:35.888823 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888711 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-cni-binary-copy\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.888823 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888742 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-multus-conf-dir\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.888823 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888770 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwpjj\" (UniqueName: \"kubernetes.io/projected/626f5874-a75d-42b5-b585-a019b92b5283-kube-api-access-kwpjj\") pod \"network-metrics-daemon-b8dk6\" (UID: \"626f5874-a75d-42b5-b585-a019b92b5283\") " pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:35.888823 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888801 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-run-systemd\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.889294 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888830 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4207b204-e57a-4d2f-b266-87d6831c6ef1-ovn-node-metrics-cert\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.889294 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888859 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2cg8\" (UniqueName: \"kubernetes.io/projected/4207b204-e57a-4d2f-b266-87d6831c6ef1-kube-api-access-r2cg8\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.889294 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888884 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.889294 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.888913 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b843783-33a8-4332-8d96-a21b5abc3a40-cnibin\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.889294 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.889090 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5b843783-33a8-4332-8d96-a21b5abc3a40-os-release\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.889294 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.889127 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b843783-33a8-4332-8d96-a21b5abc3a40-cni-binary-copy\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.889915 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.889899 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 21:21:35.891526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.890777 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-systemd\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.891526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.890816 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-multus-daemon-config\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.891526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.890838 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ee182599-bb07-4f59-bc97-1efdff4e0da8-serviceca\") pod \"node-ca-7djjr\" (UID: \"ee182599-bb07-4f59-bc97-1efdff4e0da8\") " pod="openshift-image-registry/node-ca-7djjr" Jul 06 21:21:35.891526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.890859 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-node-log\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.891526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.890921 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz5sj\" (UniqueName: \"kubernetes.io/projected/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-kube-api-access-dz5sj\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.891526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.890964 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-modprobe-d\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.891526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.890988 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee182599-bb07-4f59-bc97-1efdff4e0da8-host\") pod \"node-ca-7djjr\" (UID: \"ee182599-bb07-4f59-bc97-1efdff4e0da8\") " pod="openshift-image-registry/node-ca-7djjr" Jul 06 21:21:35.891526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891052 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-registration-dir\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.891526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891076 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-sys-fs\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.891526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891098 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-run\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.891526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891161 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-var-lib-kubelet\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.891526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891183 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-hostroot\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.891526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891228 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-etc-openvswitch\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.891526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891250 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4207b204-e57a-4d2f-b266-87d6831c6ef1-env-overrides\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891272 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b843783-33a8-4332-8d96-a21b5abc3a40-system-cni-dir\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891293 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-os-release\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891329 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-run-k8s-cni-cncf-io\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891429 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7231950-58b8-40d6-ad8d-7a57df972c1f-tmp-dir\") pod \"node-resolver-qmqnl\" (UID: \"e7231950-58b8-40d6-ad8d-7a57df972c1f\") " pod="openshift-dns/node-resolver-qmqnl" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891454 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-run-ovn\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891477 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b843783-33a8-4332-8d96-a21b5abc3a40-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891513 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-kubernetes\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891531 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891537 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8fnm\" (UniqueName: \"kubernetes.io/projected/ee182599-bb07-4f59-bc97-1efdff4e0da8-kube-api-access-m8fnm\") pod \"node-ca-7djjr\" (UID: \"ee182599-bb07-4f59-bc97-1efdff4e0da8\") " pod="openshift-image-registry/node-ca-7djjr" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891601 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-cni-bin\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891624 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4dsq\" (UniqueName: \"kubernetes.io/projected/5b843783-33a8-4332-8d96-a21b5abc3a40-kube-api-access-s4dsq\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891646 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-sysctl-d\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891668 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-run-ovn-kubernetes\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891707 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-smmrp\"" Jul 06 21:21:35.892236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891719 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 21:21:35.892919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891732 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-sysctl-conf\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.892919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891433 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 21:21:35.892919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891803 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-multus-cni-dir\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.892919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891870 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-var-lib-cni-multus\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.892919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891904 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-slash\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.892919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891972 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-socket-dir\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.892919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.891998 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-lib-modules\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.892919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892021 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gp2q\" (UniqueName: \"kubernetes.io/projected/89c932c7-a11a-4fb9-b862-fa08214a6335-kube-api-access-4gp2q\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.892919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892047 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5cc6\" (UniqueName: \"kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6\") pod \"network-check-target-zx6nz\" (UID: \"1f0cad70-ce55-4daf-95bf-43060c28dcae\") " pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:35.892919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892081 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-var-lib-kubelet\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.892919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892107 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-cni-netd\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.892919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892127 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-sys\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.892919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892168 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-multus-socket-dir-parent\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.892919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892205 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-etc-kubernetes\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.893637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892231 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e7231950-58b8-40d6-ad8d-7a57df972c1f-hosts-file\") pod \"node-resolver-qmqnl\" (UID: \"e7231950-58b8-40d6-ad8d-7a57df972c1f\") " pod="openshift-dns/node-resolver-qmqnl" Jul 06 21:21:35.893637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892256 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-run-openvswitch\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.893637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892280 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-log-socket\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.893637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892303 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.893637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892327 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-host\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.893637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892363 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-run-netns\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.893637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892389 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-run-multus-certs\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.893637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892423 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs\") pod \"network-metrics-daemon-b8dk6\" (UID: \"626f5874-a75d-42b5-b585-a019b92b5283\") " pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:35.893637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892448 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-var-lib-openvswitch\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.893637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892472 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-sysconfig\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.893637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892496 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlpzp\" (UniqueName: \"kubernetes.io/projected/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-kube-api-access-zlpzp\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.893637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892533 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-systemd-units\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.893637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892563 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4207b204-e57a-4d2f-b266-87d6831c6ef1-ovnkube-config\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.893637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892602 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-dbus\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:35.894299 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.892626 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4207b204-e57a-4d2f-b266-87d6831c6ef1-ovnkube-script-lib\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.980119 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.980087 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 21:21:35.993573 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993510 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 21:16:34 +0000 UTC" deadline="2028-04-05 16:43:07.798750722 +0000 UTC" Jul 06 21:21:35.993573 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993560 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15331h21m31.805194365s" Jul 06 21:21:35.993573 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993564 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b843783-33a8-4332-8d96-a21b5abc3a40-cnibin\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.993818 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993596 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5b843783-33a8-4332-8d96-a21b5abc3a40-os-release\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.993818 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993621 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b843783-33a8-4332-8d96-a21b5abc3a40-cni-binary-copy\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.993818 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993645 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0da29942-fe44-4e12-b3fc-ddbd8796636e-iptables-alerter-script\") pod \"iptables-alerter-9npmj\" (UID: \"0da29942-fe44-4e12-b3fc-ddbd8796636e\") " pod="openshift-network-operator/iptables-alerter-9npmj" Jul 06 21:21:35.993818 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993651 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5b843783-33a8-4332-8d96-a21b5abc3a40-cnibin\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.993818 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993663 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-systemd\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.993818 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993701 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-multus-daemon-config\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.993818 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993718 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-systemd\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.993818 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993758 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ee182599-bb07-4f59-bc97-1efdff4e0da8-serviceca\") pod \"node-ca-7djjr\" (UID: \"ee182599-bb07-4f59-bc97-1efdff4e0da8\") " pod="openshift-image-registry/node-ca-7djjr" Jul 06 21:21:35.993818 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993774 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5b843783-33a8-4332-8d96-a21b5abc3a40-os-release\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.993818 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993784 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-node-log\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993827 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz5sj\" (UniqueName: \"kubernetes.io/projected/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-kube-api-access-dz5sj\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993848 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-node-log\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993852 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-modprobe-d\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993875 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee182599-bb07-4f59-bc97-1efdff4e0da8-host\") pod \"node-ca-7djjr\" (UID: \"ee182599-bb07-4f59-bc97-1efdff4e0da8\") " pod="openshift-image-registry/node-ca-7djjr" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993901 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-registration-dir\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993969 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-sys-fs\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.993992 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-run\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994015 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-var-lib-kubelet\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994040 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-hostroot\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994062 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-etc-openvswitch\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994086 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4207b204-e57a-4d2f-b266-87d6831c6ef1-env-overrides\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994125 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b843783-33a8-4332-8d96-a21b5abc3a40-system-cni-dir\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994137 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-run\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994150 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-os-release\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.994369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994186 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-run-k8s-cni-cncf-io\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994201 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-os-release\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994214 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7231950-58b8-40d6-ad8d-7a57df972c1f-tmp-dir\") pod \"node-resolver-qmqnl\" (UID: \"e7231950-58b8-40d6-ad8d-7a57df972c1f\") " pod="openshift-dns/node-resolver-qmqnl" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994235 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-var-lib-kubelet\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994238 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-run-ovn\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994264 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-hostroot\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994265 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b843783-33a8-4332-8d96-a21b5abc3a40-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994289 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-etc-openvswitch\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994293 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-kubernetes\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994318 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8fnm\" (UniqueName: \"kubernetes.io/projected/ee182599-bb07-4f59-bc97-1efdff4e0da8-kube-api-access-m8fnm\") pod \"node-ca-7djjr\" (UID: \"ee182599-bb07-4f59-bc97-1efdff4e0da8\") " pod="openshift-image-registry/node-ca-7djjr" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994326 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5b843783-33a8-4332-8d96-a21b5abc3a40-system-cni-dir\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994341 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-cni-bin\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994352 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-run-ovn\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994369 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4dsq\" (UniqueName: \"kubernetes.io/projected/5b843783-33a8-4332-8d96-a21b5abc3a40-kube-api-access-s4dsq\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994384 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-sysctl-d\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.995074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994401 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-run-ovn-kubernetes\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.995760 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994417 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0da29942-fe44-4e12-b3fc-ddbd8796636e-host-slash\") pod \"iptables-alerter-9npmj\" (UID: \"0da29942-fe44-4e12-b3fc-ddbd8796636e\") " pod="openshift-network-operator/iptables-alerter-9npmj" Jul 06 21:21:35.995760 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994434 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-sysctl-conf\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.995760 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994448 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-multus-cni-dir\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.995760 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994464 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-var-lib-cni-multus\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.995760 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994478 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-slash\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.995760 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994483 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5b843783-33a8-4332-8d96-a21b5abc3a40-cni-binary-copy\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.995760 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994496 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-socket-dir\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.995760 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994500 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5b843783-33a8-4332-8d96-a21b5abc3a40-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.995760 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994534 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-lib-modules\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.995760 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994562 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gp2q\" (UniqueName: \"kubernetes.io/projected/89c932c7-a11a-4fb9-b862-fa08214a6335-kube-api-access-4gp2q\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.995760 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994560 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-slash\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.995760 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994608 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-var-lib-cni-multus\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.995760 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994610 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5cc6\" (UniqueName: \"kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6\") pod \"network-check-target-zx6nz\" (UID: \"1f0cad70-ce55-4daf-95bf-43060c28dcae\") " pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:35.995760 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994650 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-var-lib-kubelet\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994675 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-cni-netd\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994703 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-sys\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994727 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-multus-socket-dir-parent\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994753 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-etc-kubernetes\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994777 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e7231950-58b8-40d6-ad8d-7a57df972c1f-hosts-file\") pod \"node-resolver-qmqnl\" (UID: \"e7231950-58b8-40d6-ad8d-7a57df972c1f\") " pod="openshift-dns/node-resolver-qmqnl" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994801 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-run-openvswitch\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994824 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-log-socket\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994856 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994876 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4207b204-e57a-4d2f-b266-87d6831c6ef1-env-overrides\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994881 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-host\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994899 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-kubernetes\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994906 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-run-netns\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994915 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-registration-dir\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995028 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-socket-dir\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.996479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995046 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-multus-daemon-config\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995082 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-cni-bin\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995155 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-modprobe-d\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995168 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-sysctl-d\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995177 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-lib-modules\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995220 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-sys-fs\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995234 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-run-multus-certs\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995259 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-run-k8s-cni-cncf-io\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995266 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs\") pod \"network-metrics-daemon-b8dk6\" (UID: \"626f5874-a75d-42b5-b585-a019b92b5283\") " pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995292 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-var-lib-openvswitch\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995322 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45jqk\" (UniqueName: \"kubernetes.io/projected/0da29942-fe44-4e12-b3fc-ddbd8796636e-kube-api-access-45jqk\") pod \"iptables-alerter-9npmj\" (UID: \"0da29942-fe44-4e12-b3fc-ddbd8796636e\") " pod="openshift-network-operator/iptables-alerter-9npmj" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995349 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-sysconfig\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995373 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7231950-58b8-40d6-ad8d-7a57df972c1f-tmp-dir\") pod \"node-resolver-qmqnl\" (UID: \"e7231950-58b8-40d6-ad8d-7a57df972c1f\") " pod="openshift-dns/node-resolver-qmqnl" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995374 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlpzp\" (UniqueName: \"kubernetes.io/projected/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-kube-api-access-zlpzp\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995417 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-multus-cni-dir\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.997196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995431 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-systemd-units\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995448 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4207b204-e57a-4d2f-b266-87d6831c6ef1-ovnkube-config\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995464 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-dbus\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995481 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4207b204-e57a-4d2f-b266-87d6831c6ef1-ovnkube-script-lib\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995499 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2692f890-289e-4696-9a23-94c43e3a991c-agent-certs\") pod \"konnectivity-agent-dctfx\" (UID: \"2692f890-289e-4696-9a23-94c43e3a991c\") " pod="kube-system/konnectivity-agent-dctfx" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995516 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-tuned\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995530 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89c932c7-a11a-4fb9-b862-fa08214a6335-tmp\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995537 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-sysctl-conf\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995544 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-cnibin\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995557 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ee182599-bb07-4f59-bc97-1efdff4e0da8-serviceca\") pod \"node-ca-7djjr\" (UID: \"ee182599-bb07-4f59-bc97-1efdff4e0da8\") " pod="openshift-image-registry/node-ca-7djjr" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995576 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-cni-netd\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995577 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-run-netns\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995583 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-var-lib-kubelet\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995597 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-systemd-units\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.994562 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee182599-bb07-4f59-bc97-1efdff4e0da8-host\") pod \"node-ca-7djjr\" (UID: \"ee182599-bb07-4f59-bc97-1efdff4e0da8\") " pod="openshift-image-registry/node-ca-7djjr" Jul 06 21:21:35.997900 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995603 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-device-dir\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.998612 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995628 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5b843783-33a8-4332-8d96-a21b5abc3a40-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.998612 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995633 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-cnibin\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.998612 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995656 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5b843783-33a8-4332-8d96-a21b5abc3a40-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:35.998612 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995682 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2692f890-289e-4696-9a23-94c43e3a991c-konnectivity-ca\") pod \"konnectivity-agent-dctfx\" (UID: \"2692f890-289e-4696-9a23-94c43e3a991c\") " pod="kube-system/konnectivity-agent-dctfx" Jul 06 21:21:35.998612 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995684 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-run-multus-certs\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.998612 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995706 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-system-cni-dir\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.998612 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995729 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-var-lib-cni-bin\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.998612 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995756 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnmt2\" (UniqueName: \"kubernetes.io/projected/e7231950-58b8-40d6-ad8d-7a57df972c1f-kube-api-access-vnmt2\") pod \"node-resolver-qmqnl\" (UID: \"e7231950-58b8-40d6-ad8d-7a57df972c1f\") " pod="openshift-dns/node-resolver-qmqnl" Jul 06 21:21:35.998612 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:35.995765 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:21:35.998612 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995781 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-kubelet\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.998612 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995792 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-sys\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.998612 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995807 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.998612 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:35.995825 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs podName:626f5874-a75d-42b5-b585-a019b92b5283 nodeName:}" failed. No retries permitted until 2026-07-06 21:21:36.495805405 +0000 UTC m=+4.247428478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs") pod "network-metrics-daemon-b8dk6" (UID: "626f5874-a75d-42b5-b585-a019b92b5283") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:21:35.998612 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995852 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-multus-socket-dir-parent\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995852 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995867 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-var-lib-openvswitch\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995295 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-run-ovn-kubernetes\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995879 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4207b204-e57a-4d2f-b266-87d6831c6ef1-ovnkube-config\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995909 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e7231950-58b8-40d6-ad8d-7a57df972c1f-hosts-file\") pod \"node-resolver-qmqnl\" (UID: \"e7231950-58b8-40d6-ad8d-7a57df972c1f\") " pod="openshift-dns/node-resolver-qmqnl" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995881 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-run-netns\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995925 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-dbus\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995966 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-log-socket\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995974 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-run-openvswitch\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996015 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-host\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996019 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996051 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-run-netns\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996145 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-kubelet-config\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996177 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:35.999300 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996202 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-cni-binary-copy\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996224 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-multus-conf-dir\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996248 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwpjj\" (UniqueName: \"kubernetes.io/projected/626f5874-a75d-42b5-b585-a019b92b5283-kube-api-access-kwpjj\") pod \"network-metrics-daemon-b8dk6\" (UID: \"626f5874-a75d-42b5-b585-a019b92b5283\") " pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996272 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-run-systemd\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996293 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4207b204-e57a-4d2f-b266-87d6831c6ef1-ovnkube-script-lib\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996299 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4207b204-e57a-4d2f-b266-87d6831c6ef1-ovn-node-metrics-cert\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996346 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2cg8\" (UniqueName: \"kubernetes.io/projected/4207b204-e57a-4d2f-b266-87d6831c6ef1-kube-api-access-r2cg8\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996373 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996387 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-host-var-lib-cni-bin\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996517 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996535 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5b843783-33a8-4332-8d96-a21b5abc3a40-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996553 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-kubelet-config\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996567 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-sysconfig\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.995917 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-etc-kubernetes\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996578 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5b843783-33a8-4332-8d96-a21b5abc3a40-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:36.000017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996594 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 21:21:36.000761 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996640 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-device-dir\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:36.000761 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996687 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-multus-conf-dir\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:36.000761 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:35.996796 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 21:21:36.000761 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:35.996852 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret podName:ae8aa509-49e8-451b-8987-efcd4d6a4f1f nodeName:}" failed. No retries permitted until 2026-07-06 21:21:36.496834604 +0000 UTC m=+4.248457654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret") pod "global-pull-secret-syncer-zb6q8" (UID: "ae8aa509-49e8-451b-8987-efcd4d6a4f1f") : object "kube-system"/"original-pull-secret" not registered Jul 06 21:21:36.000761 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996854 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-system-cni-dir\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:36.000761 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996894 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-host-kubelet\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:36.000761 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996911 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2692f890-289e-4696-9a23-94c43e3a991c-konnectivity-ca\") pod \"konnectivity-agent-dctfx\" (UID: \"2692f890-289e-4696-9a23-94c43e3a991c\") " pod="kube-system/konnectivity-agent-dctfx" Jul 06 21:21:36.000761 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.996924 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4207b204-e57a-4d2f-b266-87d6831c6ef1-run-systemd\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:36.000761 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:35.997220 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-cni-binary-copy\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:36.001209 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.000868 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89c932c7-a11a-4fb9-b862-fa08214a6335-tmp\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:36.001209 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.000916 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/89c932c7-a11a-4fb9-b862-fa08214a6335-etc-tuned\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:36.001305 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.001215 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2692f890-289e-4696-9a23-94c43e3a991c-agent-certs\") pod \"konnectivity-agent-dctfx\" (UID: \"2692f890-289e-4696-9a23-94c43e3a991c\") " pod="kube-system/konnectivity-agent-dctfx" Jul 06 21:21:36.002427 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.002407 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4207b204-e57a-4d2f-b266-87d6831c6ef1-ovn-node-metrics-cert\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:36.011529 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.011491 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8fnm\" (UniqueName: \"kubernetes.io/projected/ee182599-bb07-4f59-bc97-1efdff4e0da8-kube-api-access-m8fnm\") pod \"node-ca-7djjr\" (UID: \"ee182599-bb07-4f59-bc97-1efdff4e0da8\") " pod="openshift-image-registry/node-ca-7djjr" Jul 06 21:21:36.011529 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.011516 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnmt2\" (UniqueName: \"kubernetes.io/projected/e7231950-58b8-40d6-ad8d-7a57df972c1f-kube-api-access-vnmt2\") pod \"node-resolver-qmqnl\" (UID: \"e7231950-58b8-40d6-ad8d-7a57df972c1f\") " pod="openshift-dns/node-resolver-qmqnl" Jul 06 21:21:36.017316 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:36.017294 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 21:21:36.017420 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:36.017323 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 21:21:36.017420 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:36.017337 2573 projected.go:196] Error preparing data for projected volume kube-api-access-s5cc6 for pod openshift-network-diagnostics/network-check-target-zx6nz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:21:36.017420 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:36.017398 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6 podName:1f0cad70-ce55-4daf-95bf-43060c28dcae nodeName:}" failed. No retries permitted until 2026-07-06 21:21:36.51738036 +0000 UTC m=+4.269003409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-s5cc6" (UniqueName: "kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6") pod "network-check-target-zx6nz" (UID: "1f0cad70-ce55-4daf-95bf-43060c28dcae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:21:36.020139 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.020041 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwpjj\" (UniqueName: \"kubernetes.io/projected/626f5874-a75d-42b5-b585-a019b92b5283-kube-api-access-kwpjj\") pod \"network-metrics-daemon-b8dk6\" (UID: \"626f5874-a75d-42b5-b585-a019b92b5283\") " pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:36.020780 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.020755 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlpzp\" (UniqueName: \"kubernetes.io/projected/42a478dd-c1ae-4a3b-898e-9b6386b6bb22-kube-api-access-zlpzp\") pod \"multus-p2t6l\" (UID: \"42a478dd-c1ae-4a3b-898e-9b6386b6bb22\") " pod="openshift-multus/multus-p2t6l" Jul 06 21:21:36.021066 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.021042 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2cg8\" (UniqueName: \"kubernetes.io/projected/4207b204-e57a-4d2f-b266-87d6831c6ef1-kube-api-access-r2cg8\") pod \"ovnkube-node-mfjvh\" (UID: \"4207b204-e57a-4d2f-b266-87d6831c6ef1\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:36.022685 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.022241 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gp2q\" (UniqueName: \"kubernetes.io/projected/89c932c7-a11a-4fb9-b862-fa08214a6335-kube-api-access-4gp2q\") pod \"tuned-wtb6f\" (UID: \"89c932c7-a11a-4fb9-b862-fa08214a6335\") " pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:36.022685 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.022440 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz5sj\" (UniqueName: \"kubernetes.io/projected/cd485456-58ee-4d2c-8bdd-b61cce0a42a3-kube-api-access-dz5sj\") pod \"aws-ebs-csi-driver-node-sszk6\" (UID: \"cd485456-58ee-4d2c-8bdd-b61cce0a42a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:36.022685 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.022619 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4dsq\" (UniqueName: \"kubernetes.io/projected/5b843783-33a8-4332-8d96-a21b5abc3a40-kube-api-access-s4dsq\") pod \"multus-additional-cni-plugins-vsbrr\" (UID: \"5b843783-33a8-4332-8d96-a21b5abc3a40\") " pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:36.097240 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.097206 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0da29942-fe44-4e12-b3fc-ddbd8796636e-host-slash\") pod \"iptables-alerter-9npmj\" (UID: \"0da29942-fe44-4e12-b3fc-ddbd8796636e\") " pod="openshift-network-operator/iptables-alerter-9npmj" Jul 06 21:21:36.097380 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.097294 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45jqk\" (UniqueName: \"kubernetes.io/projected/0da29942-fe44-4e12-b3fc-ddbd8796636e-kube-api-access-45jqk\") pod \"iptables-alerter-9npmj\" (UID: \"0da29942-fe44-4e12-b3fc-ddbd8796636e\") " pod="openshift-network-operator/iptables-alerter-9npmj" Jul 06 21:21:36.097380 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.097304 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0da29942-fe44-4e12-b3fc-ddbd8796636e-host-slash\") pod \"iptables-alerter-9npmj\" (UID: \"0da29942-fe44-4e12-b3fc-ddbd8796636e\") " pod="openshift-network-operator/iptables-alerter-9npmj" Jul 06 21:21:36.097380 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.097349 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0da29942-fe44-4e12-b3fc-ddbd8796636e-iptables-alerter-script\") pod \"iptables-alerter-9npmj\" (UID: \"0da29942-fe44-4e12-b3fc-ddbd8796636e\") " pod="openshift-network-operator/iptables-alerter-9npmj" Jul 06 21:21:36.098504 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.098484 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0da29942-fe44-4e12-b3fc-ddbd8796636e-iptables-alerter-script\") pod \"iptables-alerter-9npmj\" (UID: \"0da29942-fe44-4e12-b3fc-ddbd8796636e\") " pod="openshift-network-operator/iptables-alerter-9npmj" Jul 06 21:21:36.113224 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.113156 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45jqk\" (UniqueName: \"kubernetes.io/projected/0da29942-fe44-4e12-b3fc-ddbd8796636e-kube-api-access-45jqk\") pod \"iptables-alerter-9npmj\" (UID: \"0da29942-fe44-4e12-b3fc-ddbd8796636e\") " pod="openshift-network-operator/iptables-alerter-9npmj" Jul 06 21:21:36.172658 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.172627 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dctfx" Jul 06 21:21:36.186569 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.186545 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" Jul 06 21:21:36.202040 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.202008 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qmqnl" Jul 06 21:21:36.204915 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.204894 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7djjr" Jul 06 21:21:36.212911 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.212890 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:21:36.216066 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.216021 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal" event={"ID":"91fb7639492fa2b32fb2be643b9b7d5e","Type":"ContainerStarted","Data":"8902baebc44f683edffe9a8f086fe09442d7ffd98df62aadfa7b7c2459fe3d5c"} Jul 06 21:21:36.217220 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.217193 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-78.ec2.internal" event={"ID":"02bd88d5d5006b9faa87b2a2edfa6ac4","Type":"ContainerStarted","Data":"05b5dcc10874af3203f9cbb775010ae29a3a90f5c88b0e9eecac213c0cb3fafd"} Jul 06 21:21:36.220464 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.220449 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" Jul 06 21:21:36.229057 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.229038 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vsbrr" Jul 06 21:21:36.234930 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.234913 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p2t6l" Jul 06 21:21:36.241698 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.241683 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9npmj" Jul 06 21:21:36.259038 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.259014 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 21:21:36.499416 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.499347 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs\") pod \"network-metrics-daemon-b8dk6\" (UID: \"626f5874-a75d-42b5-b585-a019b92b5283\") " pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:36.499416 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.499384 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:36.499581 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:36.499490 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:21:36.499581 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:36.499545 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs podName:626f5874-a75d-42b5-b585-a019b92b5283 nodeName:}" failed. No retries permitted until 2026-07-06 21:21:37.499531224 +0000 UTC m=+5.251154266 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs") pod "network-metrics-daemon-b8dk6" (UID: "626f5874-a75d-42b5-b585-a019b92b5283") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:21:36.499581 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:36.499490 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 21:21:36.499693 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:36.499612 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret podName:ae8aa509-49e8-451b-8987-efcd4d6a4f1f nodeName:}" failed. No retries permitted until 2026-07-06 21:21:37.499600501 +0000 UTC m=+5.251223546 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret") pod "global-pull-secret-syncer-zb6q8" (UID: "ae8aa509-49e8-451b-8987-efcd4d6a4f1f") : object "kube-system"/"original-pull-secret" not registered Jul 06 21:21:36.600679 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.600645 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5cc6\" (UniqueName: \"kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6\") pod \"network-check-target-zx6nz\" (UID: \"1f0cad70-ce55-4daf-95bf-43060c28dcae\") " pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:36.600866 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:36.600790 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 21:21:36.600866 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:36.600817 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 21:21:36.600866 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:36.600831 2573 projected.go:196] Error preparing data for projected volume kube-api-access-s5cc6 for pod openshift-network-diagnostics/network-check-target-zx6nz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:21:36.601050 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:36.600889 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6 podName:1f0cad70-ce55-4daf-95bf-43060c28dcae nodeName:}" failed. No retries permitted until 2026-07-06 21:21:37.600874543 +0000 UTC m=+5.352497585 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-s5cc6" (UniqueName: "kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6") pod "network-check-target-zx6nz" (UID: "1f0cad70-ce55-4daf-95bf-43060c28dcae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:21:36.994379 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.994321 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 21:16:34 +0000 UTC" deadline="2028-02-16 15:18:32.664214725 +0000 UTC" Jul 06 21:21:36.994379 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:36.994371 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14153h56m55.669846349s" Jul 06 21:21:37.094661 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:37.094577 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89c932c7_a11a_4fb9_b862_fa08214a6335.slice/crio-17b55974360c409a38bb1e354d6db5f9ed6c759ec385990450d7c6d01c1f3f01 WatchSource:0}: Error finding container 17b55974360c409a38bb1e354d6db5f9ed6c759ec385990450d7c6d01c1f3f01: Status 404 returned error can't find the container with id 17b55974360c409a38bb1e354d6db5f9ed6c759ec385990450d7c6d01c1f3f01 Jul 06 21:21:37.096091 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:37.096067 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4207b204_e57a_4d2f_b266_87d6831c6ef1.slice/crio-630018f645790a13ba84c789b327118da07946c976f0ee35f48f4a3d2b472798 WatchSource:0}: Error finding container 630018f645790a13ba84c789b327118da07946c976f0ee35f48f4a3d2b472798: Status 404 returned error can't find the container with id 630018f645790a13ba84c789b327118da07946c976f0ee35f48f4a3d2b472798 Jul 06 21:21:37.099570 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:37.099549 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2692f890_289e_4696_9a23_94c43e3a991c.slice/crio-1e20bc10c3e2d54a8eeeb5586b2d293ec89821d9422dd927f31a77c062023da4 WatchSource:0}: Error finding container 1e20bc10c3e2d54a8eeeb5586b2d293ec89821d9422dd927f31a77c062023da4: Status 404 returned error can't find the container with id 1e20bc10c3e2d54a8eeeb5586b2d293ec89821d9422dd927f31a77c062023da4 Jul 06 21:21:37.100737 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:37.100714 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd485456_58ee_4d2c_8bdd_b61cce0a42a3.slice/crio-7b5a5adb3d3e5f70ff2ab964f8e628f31b981d1d9c1b2a24ad1a6da1bb4e7e12 WatchSource:0}: Error finding container 7b5a5adb3d3e5f70ff2ab964f8e628f31b981d1d9c1b2a24ad1a6da1bb4e7e12: Status 404 returned error can't find the container with id 7b5a5adb3d3e5f70ff2ab964f8e628f31b981d1d9c1b2a24ad1a6da1bb4e7e12 Jul 06 21:21:37.101434 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:37.101415 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da29942_fe44_4e12_b3fc_ddbd8796636e.slice/crio-71ea592e603f3b9d8f60af5d1245922d1523a432074b1fea624e38987109766a WatchSource:0}: Error finding container 71ea592e603f3b9d8f60af5d1245922d1523a432074b1fea624e38987109766a: Status 404 returned error can't find the container with id 71ea592e603f3b9d8f60af5d1245922d1523a432074b1fea624e38987109766a Jul 06 21:21:37.103063 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:37.103027 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b843783_33a8_4332_8d96_a21b5abc3a40.slice/crio-f4a91613cdeaf5326da26a8e8be320712a97dfe654c26f2c7a30295d768d67b3 WatchSource:0}: Error finding container f4a91613cdeaf5326da26a8e8be320712a97dfe654c26f2c7a30295d768d67b3: Status 404 returned error can't find the container with id f4a91613cdeaf5326da26a8e8be320712a97dfe654c26f2c7a30295d768d67b3 Jul 06 21:21:37.104195 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:37.104171 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a478dd_c1ae_4a3b_898e_9b6386b6bb22.slice/crio-460ef3be6bab5a54c3e10cbcd0b7551debfa318dd3370f2e6e00d862fedba7ec WatchSource:0}: Error finding container 460ef3be6bab5a54c3e10cbcd0b7551debfa318dd3370f2e6e00d862fedba7ec: Status 404 returned error can't find the container with id 460ef3be6bab5a54c3e10cbcd0b7551debfa318dd3370f2e6e00d862fedba7ec Jul 06 21:21:37.105538 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:37.105516 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7231950_58b8_40d6_ad8d_7a57df972c1f.slice/crio-a132db500889825770380e87144fe04868a8dc5031225028fb4a04402214b768 WatchSource:0}: Error finding container a132db500889825770380e87144fe04868a8dc5031225028fb4a04402214b768: Status 404 returned error can't find the container with id a132db500889825770380e87144fe04868a8dc5031225028fb4a04402214b768 Jul 06 21:21:37.108277 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:21:37.108254 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee182599_bb07_4f59_bc97_1efdff4e0da8.slice/crio-6b71563d6b4ccd257df68a793704470a572a9c6764e011f5e5ba230ddca5b3df WatchSource:0}: Error finding container 6b71563d6b4ccd257df68a793704470a572a9c6764e011f5e5ba230ddca5b3df: Status 404 returned error can't find the container with id 6b71563d6b4ccd257df68a793704470a572a9c6764e011f5e5ba230ddca5b3df Jul 06 21:21:37.212674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.212462 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:37.212805 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.212462 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:37.212805 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:37.212708 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:21:37.212805 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.212462 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:37.212805 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:37.212792 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:21:37.213024 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:37.212817 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:21:37.220427 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.220400 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsbrr" event={"ID":"5b843783-33a8-4332-8d96-a21b5abc3a40","Type":"ContainerStarted","Data":"f4a91613cdeaf5326da26a8e8be320712a97dfe654c26f2c7a30295d768d67b3"} Jul 06 21:21:37.224241 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.224221 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p2t6l" event={"ID":"42a478dd-c1ae-4a3b-898e-9b6386b6bb22","Type":"ContainerStarted","Data":"460ef3be6bab5a54c3e10cbcd0b7551debfa318dd3370f2e6e00d862fedba7ec"} Jul 06 21:21:37.225351 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.225330 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" event={"ID":"89c932c7-a11a-4fb9-b862-fa08214a6335","Type":"ContainerStarted","Data":"17b55974360c409a38bb1e354d6db5f9ed6c759ec385990450d7c6d01c1f3f01"} Jul 06 21:21:37.226467 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.226445 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" event={"ID":"cd485456-58ee-4d2c-8bdd-b61cce0a42a3","Type":"ContainerStarted","Data":"7b5a5adb3d3e5f70ff2ab964f8e628f31b981d1d9c1b2a24ad1a6da1bb4e7e12"} Jul 06 21:21:37.227414 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.227396 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qmqnl" event={"ID":"e7231950-58b8-40d6-ad8d-7a57df972c1f","Type":"ContainerStarted","Data":"a132db500889825770380e87144fe04868a8dc5031225028fb4a04402214b768"} Jul 06 21:21:37.229448 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.229424 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9npmj" event={"ID":"0da29942-fe44-4e12-b3fc-ddbd8796636e","Type":"ContainerStarted","Data":"71ea592e603f3b9d8f60af5d1245922d1523a432074b1fea624e38987109766a"} Jul 06 21:21:37.230991 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.230963 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dctfx" event={"ID":"2692f890-289e-4696-9a23-94c43e3a991c","Type":"ContainerStarted","Data":"1e20bc10c3e2d54a8eeeb5586b2d293ec89821d9422dd927f31a77c062023da4"} Jul 06 21:21:37.232742 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.232724 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" event={"ID":"4207b204-e57a-4d2f-b266-87d6831c6ef1","Type":"ContainerStarted","Data":"630018f645790a13ba84c789b327118da07946c976f0ee35f48f4a3d2b472798"} Jul 06 21:21:37.235974 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.235937 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-78.ec2.internal" event={"ID":"02bd88d5d5006b9faa87b2a2edfa6ac4","Type":"ContainerStarted","Data":"57903acc48cab73f9a8cb0b0939a7a343b9f5486b50fbee2a761a682baeea56f"} Jul 06 21:21:37.238120 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.238095 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7djjr" event={"ID":"ee182599-bb07-4f59-bc97-1efdff4e0da8","Type":"ContainerStarted","Data":"6b71563d6b4ccd257df68a793704470a572a9c6764e011f5e5ba230ddca5b3df"} Jul 06 21:21:37.252448 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.252406 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-78.ec2.internal" podStartSLOduration=3.252376314 podStartE2EDuration="3.252376314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:21:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:21:37.252190109 +0000 UTC m=+5.003813170" watchObservedRunningTime="2026-07-06 21:21:37.252376314 +0000 UTC m=+5.003999377" Jul 06 21:21:37.506521 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.506189 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:37.506521 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.506282 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs\") pod \"network-metrics-daemon-b8dk6\" (UID: \"626f5874-a75d-42b5-b585-a019b92b5283\") " pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:37.506521 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:37.506352 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 21:21:37.506521 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:37.506403 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:21:37.506521 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:37.506427 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret podName:ae8aa509-49e8-451b-8987-efcd4d6a4f1f nodeName:}" failed. No retries permitted until 2026-07-06 21:21:39.506407779 +0000 UTC m=+7.258030824 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret") pod "global-pull-secret-syncer-zb6q8" (UID: "ae8aa509-49e8-451b-8987-efcd4d6a4f1f") : object "kube-system"/"original-pull-secret" not registered Jul 06 21:21:37.506521 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:37.506453 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs podName:626f5874-a75d-42b5-b585-a019b92b5283 nodeName:}" failed. No retries permitted until 2026-07-06 21:21:39.50643636 +0000 UTC m=+7.258059417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs") pod "network-metrics-daemon-b8dk6" (UID: "626f5874-a75d-42b5-b585-a019b92b5283") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:21:37.607389 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.607273 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5cc6\" (UniqueName: \"kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6\") pod \"network-check-target-zx6nz\" (UID: \"1f0cad70-ce55-4daf-95bf-43060c28dcae\") " pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:37.607564 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:37.607475 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 21:21:37.607564 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:37.607501 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 21:21:37.607564 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:37.607515 2573 projected.go:196] Error preparing data for projected volume kube-api-access-s5cc6 for pod openshift-network-diagnostics/network-check-target-zx6nz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:21:37.607716 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:37.607578 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6 podName:1f0cad70-ce55-4daf-95bf-43060c28dcae nodeName:}" failed. No retries permitted until 2026-07-06 21:21:39.607558991 +0000 UTC m=+7.359182037 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-s5cc6" (UniqueName: "kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6") pod "network-check-target-zx6nz" (UID: "1f0cad70-ce55-4daf-95bf-43060c28dcae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:21:37.983619 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:37.983541 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 21:21:38.265282 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:38.265203 2573 generic.go:356] "Generic (PLEG): container finished" podID="91fb7639492fa2b32fb2be643b9b7d5e" containerID="9e0a3f58b262b1384a2f767547f05fd8cc6581764ab57e5eb586c71fa0c5cd18" exitCode=0 Jul 06 21:21:38.265827 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:38.265361 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal" event={"ID":"91fb7639492fa2b32fb2be643b9b7d5e","Type":"ContainerDied","Data":"9e0a3f58b262b1384a2f767547f05fd8cc6581764ab57e5eb586c71fa0c5cd18"} Jul 06 21:21:39.212771 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:39.211955 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:39.212771 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:39.212087 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:21:39.212771 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:39.212474 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:39.212771 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:39.212564 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:21:39.212771 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:39.212627 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:39.212771 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:39.212708 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:21:39.284865 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:39.284216 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal" event={"ID":"91fb7639492fa2b32fb2be643b9b7d5e","Type":"ContainerStarted","Data":"a75659dadf1302d31a622efbc49bcf28e2dc6648cc7bf42ed0129964dc6416ec"} Jul 06 21:21:39.525542 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:39.525440 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs\") pod \"network-metrics-daemon-b8dk6\" (UID: \"626f5874-a75d-42b5-b585-a019b92b5283\") " pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:39.525542 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:39.525526 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:39.525761 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:39.525666 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 21:21:39.525761 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:39.525732 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret podName:ae8aa509-49e8-451b-8987-efcd4d6a4f1f nodeName:}" failed. No retries permitted until 2026-07-06 21:21:43.525714004 +0000 UTC m=+11.277337047 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret") pod "global-pull-secret-syncer-zb6q8" (UID: "ae8aa509-49e8-451b-8987-efcd4d6a4f1f") : object "kube-system"/"original-pull-secret" not registered Jul 06 21:21:39.526418 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:39.526129 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:21:39.526418 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:39.526183 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs podName:626f5874-a75d-42b5-b585-a019b92b5283 nodeName:}" failed. No retries permitted until 2026-07-06 21:21:43.52616747 +0000 UTC m=+11.277790514 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs") pod "network-metrics-daemon-b8dk6" (UID: "626f5874-a75d-42b5-b585-a019b92b5283") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:21:39.626541 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:39.626507 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5cc6\" (UniqueName: \"kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6\") pod \"network-check-target-zx6nz\" (UID: \"1f0cad70-ce55-4daf-95bf-43060c28dcae\") " pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:39.626750 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:39.626732 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 21:21:39.626837 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:39.626760 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 21:21:39.626837 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:39.626773 2573 projected.go:196] Error preparing data for projected volume kube-api-access-s5cc6 for pod openshift-network-diagnostics/network-check-target-zx6nz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:21:39.626837 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:39.626827 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6 podName:1f0cad70-ce55-4daf-95bf-43060c28dcae nodeName:}" failed. No retries permitted until 2026-07-06 21:21:43.626810211 +0000 UTC m=+11.378433265 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-s5cc6" (UniqueName: "kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6") pod "network-check-target-zx6nz" (UID: "1f0cad70-ce55-4daf-95bf-43060c28dcae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:21:41.212323 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:41.212287 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:41.212776 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:41.212431 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:21:41.212853 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:41.212780 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:41.212904 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:41.212855 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:21:41.212988 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:41.212909 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:41.213040 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:41.213017 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:21:43.212096 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:43.211992 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:43.212564 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:43.212117 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:21:43.212564 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:43.212175 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:43.212564 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:43.212297 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:21:43.212564 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:43.212367 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:43.212564 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:43.212445 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:21:43.559816 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:43.559042 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs\") pod \"network-metrics-daemon-b8dk6\" (UID: \"626f5874-a75d-42b5-b585-a019b92b5283\") " pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:43.559816 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:43.559098 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:43.559816 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:43.559228 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 21:21:43.559816 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:43.559290 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret podName:ae8aa509-49e8-451b-8987-efcd4d6a4f1f nodeName:}" failed. No retries permitted until 2026-07-06 21:21:51.559272617 +0000 UTC m=+19.310895664 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret") pod "global-pull-secret-syncer-zb6q8" (UID: "ae8aa509-49e8-451b-8987-efcd4d6a4f1f") : object "kube-system"/"original-pull-secret" not registered Jul 06 21:21:43.559816 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:43.559684 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:21:43.559816 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:43.559737 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs podName:626f5874-a75d-42b5-b585-a019b92b5283 nodeName:}" failed. No retries permitted until 2026-07-06 21:21:51.559721137 +0000 UTC m=+19.311344185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs") pod "network-metrics-daemon-b8dk6" (UID: "626f5874-a75d-42b5-b585-a019b92b5283") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:21:43.660020 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:43.659938 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5cc6\" (UniqueName: \"kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6\") pod \"network-check-target-zx6nz\" (UID: \"1f0cad70-ce55-4daf-95bf-43060c28dcae\") " pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:43.660185 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:43.660148 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 21:21:43.660185 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:43.660170 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 21:21:43.660185 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:43.660181 2573 projected.go:196] Error preparing data for projected volume kube-api-access-s5cc6 for pod openshift-network-diagnostics/network-check-target-zx6nz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:21:43.660336 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:43.660235 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6 podName:1f0cad70-ce55-4daf-95bf-43060c28dcae nodeName:}" failed. No retries permitted until 2026-07-06 21:21:51.660217228 +0000 UTC m=+19.411840272 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-s5cc6" (UniqueName: "kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6") pod "network-check-target-zx6nz" (UID: "1f0cad70-ce55-4daf-95bf-43060c28dcae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:21:45.212473 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:45.212017 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:45.212473 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:45.212149 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:21:45.212473 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:45.212406 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:45.212997 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:45.212528 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:21:45.212997 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:45.212580 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:45.212997 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:45.212660 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:21:47.211876 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:47.211839 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:47.211876 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:47.211856 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:47.212382 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:47.211849 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:47.212382 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:47.211978 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:21:47.212382 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:47.212082 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:21:47.212382 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:47.212161 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:21:49.211827 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:49.211757 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:49.211827 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:49.211792 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:49.212433 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:49.211887 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:21:49.212433 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:49.211932 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:21:49.212433 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:49.211998 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:49.212433 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:49.212091 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:21:51.212424 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:51.212396 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:51.212424 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:51.212415 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:51.212858 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:51.212402 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:51.212858 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:51.212489 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:21:51.212858 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:51.212562 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:21:51.212858 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:51.212636 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:21:51.619995 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:51.619891 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs\") pod \"network-metrics-daemon-b8dk6\" (UID: \"626f5874-a75d-42b5-b585-a019b92b5283\") " pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:51.619995 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:51.619940 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:51.620193 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:51.620062 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 21:21:51.620193 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:51.620064 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:21:51.620193 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:51.620117 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret podName:ae8aa509-49e8-451b-8987-efcd4d6a4f1f nodeName:}" failed. No retries permitted until 2026-07-06 21:22:07.620098581 +0000 UTC m=+35.371721622 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret") pod "global-pull-secret-syncer-zb6q8" (UID: "ae8aa509-49e8-451b-8987-efcd4d6a4f1f") : object "kube-system"/"original-pull-secret" not registered Jul 06 21:21:51.620193 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:51.620137 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs podName:626f5874-a75d-42b5-b585-a019b92b5283 nodeName:}" failed. No retries permitted until 2026-07-06 21:22:07.620126694 +0000 UTC m=+35.371749739 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs") pod "network-metrics-daemon-b8dk6" (UID: "626f5874-a75d-42b5-b585-a019b92b5283") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:21:51.720857 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:51.720826 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5cc6\" (UniqueName: \"kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6\") pod \"network-check-target-zx6nz\" (UID: \"1f0cad70-ce55-4daf-95bf-43060c28dcae\") " pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:51.721053 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:51.720998 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 21:21:51.721053 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:51.721021 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 21:21:51.721053 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:51.721032 2573 projected.go:196] Error preparing data for projected volume kube-api-access-s5cc6 for pod openshift-network-diagnostics/network-check-target-zx6nz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:21:51.721208 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:51.721091 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6 podName:1f0cad70-ce55-4daf-95bf-43060c28dcae nodeName:}" failed. No retries permitted until 2026-07-06 21:22:07.721071699 +0000 UTC m=+35.472694742 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-s5cc6" (UniqueName: "kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6") pod "network-check-target-zx6nz" (UID: "1f0cad70-ce55-4daf-95bf-43060c28dcae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:21:53.211693 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:53.211662 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:53.212153 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:53.211662 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:53.212153 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:53.211778 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:21:53.212153 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:53.211665 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:53.212153 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:53.211881 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:21:53.212153 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:53.211980 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:21:55.212426 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:55.212386 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:55.212827 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:55.212505 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:55.212827 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:55.212514 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:55.212827 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:55.212626 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:21:55.212827 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:55.212509 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:21:55.212827 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:55.212729 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:21:56.318543 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.318359 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" event={"ID":"cd485456-58ee-4d2c-8bdd-b61cce0a42a3","Type":"ContainerStarted","Data":"a52b6fcb52b459051dbe1436a123f86d7b1a011f24c849cc32510bbbac8e8529"} Jul 06 21:21:56.319676 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.319653 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qmqnl" event={"ID":"e7231950-58b8-40d6-ad8d-7a57df972c1f","Type":"ContainerStarted","Data":"52d8c33579204ace657f7216251517dc9172ababa289329c658185d9262d0fe6"} Jul 06 21:21:56.320972 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.320928 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dctfx" event={"ID":"2692f890-289e-4696-9a23-94c43e3a991c","Type":"ContainerStarted","Data":"0280ae56bf342f1c73a0d6bc6eb33c3b0a488954121abcfcdc83ce4618f2e58b"} Jul 06 21:21:56.322284 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.322268 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:21:56.322561 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.322541 2573 generic.go:356] "Generic (PLEG): container finished" podID="4207b204-e57a-4d2f-b266-87d6831c6ef1" containerID="5fe0319fa473b23896e1afb02fc31dcbbec7979fb5928b2a37ffaf83888f059e" exitCode=1 Jul 06 21:21:56.322638 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.322606 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" event={"ID":"4207b204-e57a-4d2f-b266-87d6831c6ef1","Type":"ContainerDied","Data":"5fe0319fa473b23896e1afb02fc31dcbbec7979fb5928b2a37ffaf83888f059e"} Jul 06 21:21:56.322681 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.322660 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" event={"ID":"4207b204-e57a-4d2f-b266-87d6831c6ef1","Type":"ContainerStarted","Data":"1281c42860a32a14ee98bb5a8d164bfb66cbd31df3e8b1560e22d384af513e74"} Jul 06 21:21:56.323802 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.323784 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7djjr" event={"ID":"ee182599-bb07-4f59-bc97-1efdff4e0da8","Type":"ContainerStarted","Data":"2f406c72a61a452a960a2b5bb6db3e83b6cd724971e66ffb0f30f00cf8acd05f"} Jul 06 21:21:56.325284 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.325260 2573 generic.go:356] "Generic (PLEG): container finished" podID="5b843783-33a8-4332-8d96-a21b5abc3a40" containerID="3e1c078f4f4801ad0ed15705c968791be30622a25ca5c9c942d5eb111e1ecaa9" exitCode=0 Jul 06 21:21:56.325395 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.325337 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsbrr" event={"ID":"5b843783-33a8-4332-8d96-a21b5abc3a40","Type":"ContainerDied","Data":"3e1c078f4f4801ad0ed15705c968791be30622a25ca5c9c942d5eb111e1ecaa9"} Jul 06 21:21:56.326803 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.326778 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p2t6l" event={"ID":"42a478dd-c1ae-4a3b-898e-9b6386b6bb22","Type":"ContainerStarted","Data":"1de84a26482ee2f30af123c089759bbf2a6e05d55c225cd64e8c4d33cf1e07ea"} Jul 06 21:21:56.328447 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.328423 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" event={"ID":"89c932c7-a11a-4fb9-b862-fa08214a6335","Type":"ContainerStarted","Data":"83d8e4a7496653e7a0a1de39490ede14dd05cf169e9827d370e087fe8d23ba7e"} Jul 06 21:21:56.337588 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.337552 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qmqnl" podStartSLOduration=3.693016615 podStartE2EDuration="22.337541859s" totalImagesPullingTime="18.644525244s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:21:56.337255524 +0000 UTC m=+24.088878633" watchObservedRunningTime="2026-07-06 21:21:56.337541859 +0000 UTC m=+24.089164922" Jul 06 21:21:56.337995 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.337968 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-78.ec2.internal" podStartSLOduration=22.33793572 podStartE2EDuration="22.33793572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:21:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:21:39.306101837 +0000 UTC m=+7.057724902" watchObservedRunningTime="2026-07-06 21:21:56.33793572 +0000 UTC m=+24.089558784" Jul 06 21:21:56.353114 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.353077 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dctfx" podStartSLOduration=12.065348164 podStartE2EDuration="22.353062429s" totalImagesPullingTime="10.287714265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:21:56.352996216 +0000 UTC m=+24.104619280" watchObservedRunningTime="2026-07-06 21:21:56.353062429 +0000 UTC m=+24.104685496" Jul 06 21:21:56.371832 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.371704 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wtb6f" podStartSLOduration=3.712693035 podStartE2EDuration="22.371691795s" totalImagesPullingTime="18.65899876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:21:56.371656957 +0000 UTC m=+24.123280023" watchObservedRunningTime="2026-07-06 21:21:56.371691795 +0000 UTC m=+24.123314859" Jul 06 21:21:56.438382 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.438343 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7djjr" podStartSLOduration=3.795419204 podStartE2EDuration="22.438328046s" totalImagesPullingTime="18.642908842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:21:56.418003423 +0000 UTC m=+24.169626488" watchObservedRunningTime="2026-07-06 21:21:56.438328046 +0000 UTC m=+24.189951110" Jul 06 21:21:56.438851 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:56.438818 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-p2t6l" podStartSLOduration=3.777863608 podStartE2EDuration="22.438807318s" totalImagesPullingTime="18.66094371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:21:56.438264852 +0000 UTC m=+24.189887919" watchObservedRunningTime="2026-07-06 21:21:56.438807318 +0000 UTC m=+24.190430382" Jul 06 21:21:57.181455 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:57.181435 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 21:21:57.211992 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:57.211969 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:57.212120 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:57.211969 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:57.212120 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:57.212086 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:21:57.212233 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:57.212164 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:21:57.212233 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:57.211977 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:57.212388 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:57.212242 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:21:57.334209 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:57.334139 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:21:57.334749 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:57.334554 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" event={"ID":"4207b204-e57a-4d2f-b266-87d6831c6ef1","Type":"ContainerStarted","Data":"9d96c945e766633e3e46758df9d3082dca263f93b3bdcb55d63eec939f228c80"} Jul 06 21:21:57.334749 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:57.334618 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" event={"ID":"4207b204-e57a-4d2f-b266-87d6831c6ef1","Type":"ContainerStarted","Data":"17d654a5d9b57be7569d3edb071746085090563c54ecd5ae799ebbe7c413f14b"} Jul 06 21:21:57.337932 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:57.334764 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" event={"ID":"4207b204-e57a-4d2f-b266-87d6831c6ef1","Type":"ContainerStarted","Data":"5a496411ac164263cb6c6e6db1a96301f22530ecf940a6cb891e6db89e740178"} Jul 06 21:21:57.337932 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:57.335158 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" event={"ID":"4207b204-e57a-4d2f-b266-87d6831c6ef1","Type":"ContainerStarted","Data":"0c33d87339e34a19fe7f61770a48844479e811ef36946b6cbddc8b40f2991f15"} Jul 06 21:21:57.338821 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:57.338796 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" event={"ID":"cd485456-58ee-4d2c-8bdd-b61cce0a42a3","Type":"ContainerStarted","Data":"d48bb634d9b814de1cd229ec0b3a3d006bcfa5451e05a495efa580bfb4a7c691"} Jul 06 21:21:57.340398 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:57.340371 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9npmj" event={"ID":"0da29942-fe44-4e12-b3fc-ddbd8796636e","Type":"ContainerStarted","Data":"ba539543f6b04dcf76ec0aab2a6daeb330a4cefe1fb21a34f12603a3b39831d9"} Jul 06 21:21:57.365698 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:57.365646 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9npmj" podStartSLOduration=4.715210635 podStartE2EDuration="23.365628651s" totalImagesPullingTime="18.650418016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:21:57.364800017 +0000 UTC m=+25.116423082" watchObservedRunningTime="2026-07-06 21:21:57.365628651 +0000 UTC m=+25.117251716" Jul 06 21:21:58.115487 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:58.115390 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T21:21:57.181452782Z","UUID":"15566a0f-a382-40e9-be8d-0637d5d30f43","Handler":null,"Name":"","Endpoint":""} Jul 06 21:21:58.118088 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:58.118068 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 21:21:58.118204 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:58.118104 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 21:21:58.344089 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:58.344061 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" event={"ID":"cd485456-58ee-4d2c-8bdd-b61cce0a42a3","Type":"ContainerStarted","Data":"cd12b8c76de06ccb6a30644241161dbb70fbf8aa961dee761d254b31b2c65a20"} Jul 06 21:21:58.367526 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:58.367457 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sszk6" podStartSLOduration=3.61210496 podStartE2EDuration="24.367443085s" totalImagesPullingTime="20.755338125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:21:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:21:58.367360515 +0000 UTC m=+26.118983579" watchObservedRunningTime="2026-07-06 21:21:58.367443085 +0000 UTC m=+26.119066145" Jul 06 21:21:59.212088 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:59.211859 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:21:59.212221 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:59.211909 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:21:59.212221 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:59.212118 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:21:59.212293 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:59.212211 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:21:59.212293 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:59.211911 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:21:59.212352 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:21:59.212291 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:21:59.349109 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:59.349083 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:21:59.349500 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:21:59.349476 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" event={"ID":"4207b204-e57a-4d2f-b266-87d6831c6ef1","Type":"ContainerStarted","Data":"4ba5c448372bb97a01122fbe6f4e7e87a2e16c60156f5a8f00d2fe56c5e6b21a"} Jul 06 21:22:01.120036 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:01.119994 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dctfx" Jul 06 21:22:01.120676 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:01.120564 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dctfx" Jul 06 21:22:01.211708 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:01.211684 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:22:01.211882 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:01.211775 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:22:01.211882 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:01.211788 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:22:01.211882 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:01.211784 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:22:01.212061 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:01.211877 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:22:01.212061 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:01.211975 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:22:02.358067 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:02.357786 2573 generic.go:356] "Generic (PLEG): container finished" podID="5b843783-33a8-4332-8d96-a21b5abc3a40" containerID="ce8af51a71933a4e30a71ed83cb32edec9446b0c0249960f72239f39fba2a885" exitCode=0 Jul 06 21:22:02.358919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:02.357832 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsbrr" event={"ID":"5b843783-33a8-4332-8d96-a21b5abc3a40","Type":"ContainerDied","Data":"ce8af51a71933a4e30a71ed83cb32edec9446b0c0249960f72239f39fba2a885"} Jul 06 21:22:02.361694 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:02.361675 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:22:02.362029 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:02.362009 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" event={"ID":"4207b204-e57a-4d2f-b266-87d6831c6ef1","Type":"ContainerStarted","Data":"ab59d1157f840641e90dd35e3b5d3770e7ea36e2308c4812b61355af20c683a0"} Jul 06 21:22:02.362332 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:02.362310 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:22:02.362411 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:02.362399 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:22:02.362472 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:02.362457 2573 scope.go:117] "RemoveContainer" containerID="5fe0319fa473b23896e1afb02fc31dcbbec7979fb5928b2a37ffaf83888f059e" Jul 06 21:22:02.378461 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:02.378440 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:22:03.212086 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:03.212059 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:22:03.212244 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:03.212067 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:22:03.212244 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:03.212156 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:22:03.212244 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:03.212179 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:22:03.212392 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:03.212259 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:22:03.212392 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:03.212344 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:22:03.368642 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:03.368489 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:22:03.369086 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:03.369056 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" event={"ID":"4207b204-e57a-4d2f-b266-87d6831c6ef1","Type":"ContainerStarted","Data":"f80db7ba520d89e62f22bd6af21b9a5ab372bf2ea5ed59d8f5b943bb6e15164e"} Jul 06 21:22:03.369361 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:03.369338 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:22:03.392013 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:03.391990 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:22:03.431837 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:03.431806 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zb6q8"] Jul 06 21:22:03.432031 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:03.431896 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:22:03.432031 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:03.431991 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:22:03.441079 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:03.441054 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zx6nz"] Jul 06 21:22:03.441202 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:03.441157 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:22:03.441286 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:03.441257 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:22:03.450188 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:03.450150 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" podStartSLOduration=10.511725449 podStartE2EDuration="29.450135164s" totalImagesPullingTime="18.938409715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:22:03.448585098 +0000 UTC m=+31.200208159" watchObservedRunningTime="2026-07-06 21:22:03.450135164 +0000 UTC m=+31.201758221" Jul 06 21:22:03.473490 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:03.473438 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b8dk6"] Jul 06 21:22:03.473576 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:03.473517 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:22:03.473623 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:03.473590 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:22:04.373067 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:04.373032 2573 generic.go:356] "Generic (PLEG): container finished" podID="5b843783-33a8-4332-8d96-a21b5abc3a40" containerID="130cd5922425e4243ccd8890da3156d6b7a0dd2a1fe35d3f319296a4b4cb9ac4" exitCode=0 Jul 06 21:22:04.373486 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:04.373121 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsbrr" event={"ID":"5b843783-33a8-4332-8d96-a21b5abc3a40","Type":"ContainerDied","Data":"130cd5922425e4243ccd8890da3156d6b7a0dd2a1fe35d3f319296a4b4cb9ac4"} Jul 06 21:22:05.211711 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:05.211680 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:22:05.211829 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:05.211762 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:22:05.211829 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:05.211779 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:22:05.211895 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:05.211851 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:22:05.211934 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:05.211917 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:22:05.212065 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:05.212037 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:22:05.378131 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:05.378062 2573 generic.go:356] "Generic (PLEG): container finished" podID="5b843783-33a8-4332-8d96-a21b5abc3a40" containerID="d7d5ead54b4b0cfbffb9ea63c795516d23c247ddfc18d1988134b40a9fba4e44" exitCode=0 Jul 06 21:22:05.378586 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:05.378147 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsbrr" event={"ID":"5b843783-33a8-4332-8d96-a21b5abc3a40","Type":"ContainerDied","Data":"d7d5ead54b4b0cfbffb9ea63c795516d23c247ddfc18d1988134b40a9fba4e44"} Jul 06 21:22:05.623232 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:05.623211 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dctfx" Jul 06 21:22:05.623349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:05.623339 2573 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 21:22:05.623786 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:05.623767 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dctfx" Jul 06 21:22:07.212327 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:07.212295 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:22:07.212922 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:07.212407 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:22:07.212922 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:07.212414 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:22:07.212922 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:07.212422 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:22:07.212922 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:07.212506 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:22:07.212922 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:07.212588 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:22:07.644049 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:07.644015 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs\") pod \"network-metrics-daemon-b8dk6\" (UID: \"626f5874-a75d-42b5-b585-a019b92b5283\") " pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:22:07.644241 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:07.644068 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:22:07.644241 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:07.644172 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 21:22:07.644241 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:07.644232 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret podName:ae8aa509-49e8-451b-8987-efcd4d6a4f1f nodeName:}" failed. No retries permitted until 2026-07-06 21:22:39.644215101 +0000 UTC m=+67.395838157 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret") pod "global-pull-secret-syncer-zb6q8" (UID: "ae8aa509-49e8-451b-8987-efcd4d6a4f1f") : object "kube-system"/"original-pull-secret" not registered Jul 06 21:22:07.644393 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:07.644174 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:22:07.644393 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:07.644315 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs podName:626f5874-a75d-42b5-b585-a019b92b5283 nodeName:}" failed. No retries permitted until 2026-07-06 21:22:39.644294854 +0000 UTC m=+67.395917907 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs") pod "network-metrics-daemon-b8dk6" (UID: "626f5874-a75d-42b5-b585-a019b92b5283") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 21:22:07.745178 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:07.745150 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5cc6\" (UniqueName: \"kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6\") pod \"network-check-target-zx6nz\" (UID: \"1f0cad70-ce55-4daf-95bf-43060c28dcae\") " pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:22:07.745351 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:07.745296 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 21:22:07.745351 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:07.745316 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 21:22:07.745351 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:07.745327 2573 projected.go:196] Error preparing data for projected volume kube-api-access-s5cc6 for pod openshift-network-diagnostics/network-check-target-zx6nz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:22:07.745505 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:07.745383 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6 podName:1f0cad70-ce55-4daf-95bf-43060c28dcae nodeName:}" failed. No retries permitted until 2026-07-06 21:22:39.745365243 +0000 UTC m=+67.496988288 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-s5cc6" (UniqueName: "kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6") pod "network-check-target-zx6nz" (UID: "1f0cad70-ce55-4daf-95bf-43060c28dcae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 21:22:09.212280 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.212243 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:22:09.212706 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.212243 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:22:09.212706 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:09.212379 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8dk6" podUID="626f5874-a75d-42b5-b585-a019b92b5283" Jul 06 21:22:09.212706 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.212248 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:22:09.212706 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:09.212417 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zx6nz" podUID="1f0cad70-ce55-4daf-95bf-43060c28dcae" Jul 06 21:22:09.212706 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:09.212536 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zb6q8" podUID="ae8aa509-49e8-451b-8987-efcd4d6a4f1f" Jul 06 21:22:09.538700 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.538508 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-78.ec2.internal" event="NodeReady" Jul 06 21:22:09.538872 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.538839 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 21:22:09.596023 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.595988 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qdv89"] Jul 06 21:22:09.598510 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.598488 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:09.602933 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.602509 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 21:22:09.602933 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.602775 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9fg2d\"" Jul 06 21:22:09.603137 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.603103 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 21:22:09.604442 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.604414 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-r2mds"] Jul 06 21:22:09.606352 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.606335 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r2mds" Jul 06 21:22:09.608763 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.608741 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 21:22:09.609316 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.609300 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 21:22:09.609464 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.609446 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-v7k9x\"" Jul 06 21:22:09.609532 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.609524 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 21:22:09.611830 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.611803 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qdv89"] Jul 06 21:22:09.623276 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.623249 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r2mds"] Jul 06 21:22:09.760985 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.760938 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8b6p\" (UniqueName: \"kubernetes.io/projected/96f500f5-8245-4d8b-98f4-60c5ff8821ef-kube-api-access-s8b6p\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:09.760985 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.760988 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/96f500f5-8245-4d8b-98f4-60c5ff8821ef-config-volume\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:09.761213 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.761033 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96f500f5-8245-4d8b-98f4-60c5ff8821ef-tmp-dir\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:09.761213 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.761109 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/594d6a6c-c70c-4642-9c6b-3745fbd64c4f-cert\") pod \"ingress-canary-r2mds\" (UID: \"594d6a6c-c70c-4642-9c6b-3745fbd64c4f\") " pod="openshift-ingress-canary/ingress-canary-r2mds" Jul 06 21:22:09.761213 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.761135 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqtdg\" (UniqueName: \"kubernetes.io/projected/594d6a6c-c70c-4642-9c6b-3745fbd64c4f-kube-api-access-xqtdg\") pod \"ingress-canary-r2mds\" (UID: \"594d6a6c-c70c-4642-9c6b-3745fbd64c4f\") " pod="openshift-ingress-canary/ingress-canary-r2mds" Jul 06 21:22:09.761213 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.761164 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:09.861739 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.861639 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:09.861739 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.861700 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8b6p\" (UniqueName: \"kubernetes.io/projected/96f500f5-8245-4d8b-98f4-60c5ff8821ef-kube-api-access-s8b6p\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:09.861739 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.861728 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/96f500f5-8245-4d8b-98f4-60c5ff8821ef-config-volume\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:09.862002 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.861753 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96f500f5-8245-4d8b-98f4-60c5ff8821ef-tmp-dir\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:09.862002 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:09.861798 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 21:22:09.862002 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:09.861881 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls podName:96f500f5-8245-4d8b-98f4-60c5ff8821ef nodeName:}" failed. No retries permitted until 2026-07-06 21:22:10.361860369 +0000 UTC m=+38.113483428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls") pod "dns-default-qdv89" (UID: "96f500f5-8245-4d8b-98f4-60c5ff8821ef") : secret "dns-default-metrics-tls" not found Jul 06 21:22:09.862169 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.862033 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/594d6a6c-c70c-4642-9c6b-3745fbd64c4f-cert\") pod \"ingress-canary-r2mds\" (UID: \"594d6a6c-c70c-4642-9c6b-3745fbd64c4f\") " pod="openshift-ingress-canary/ingress-canary-r2mds" Jul 06 21:22:09.862169 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.862084 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqtdg\" (UniqueName: \"kubernetes.io/projected/594d6a6c-c70c-4642-9c6b-3745fbd64c4f-kube-api-access-xqtdg\") pod \"ingress-canary-r2mds\" (UID: \"594d6a6c-c70c-4642-9c6b-3745fbd64c4f\") " pod="openshift-ingress-canary/ingress-canary-r2mds" Jul 06 21:22:09.862169 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.862103 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96f500f5-8245-4d8b-98f4-60c5ff8821ef-tmp-dir\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:09.862450 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.862429 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/96f500f5-8245-4d8b-98f4-60c5ff8821ef-config-volume\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:09.866340 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.866311 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/594d6a6c-c70c-4642-9c6b-3745fbd64c4f-cert\") pod \"ingress-canary-r2mds\" (UID: \"594d6a6c-c70c-4642-9c6b-3745fbd64c4f\") " pod="openshift-ingress-canary/ingress-canary-r2mds" Jul 06 21:22:09.873215 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.873193 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8b6p\" (UniqueName: \"kubernetes.io/projected/96f500f5-8245-4d8b-98f4-60c5ff8821ef-kube-api-access-s8b6p\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:09.874640 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.874613 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqtdg\" (UniqueName: \"kubernetes.io/projected/594d6a6c-c70c-4642-9c6b-3745fbd64c4f-kube-api-access-xqtdg\") pod \"ingress-canary-r2mds\" (UID: \"594d6a6c-c70c-4642-9c6b-3745fbd64c4f\") " pod="openshift-ingress-canary/ingress-canary-r2mds" Jul 06 21:22:09.920145 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:09.920117 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r2mds" Jul 06 21:22:10.131292 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:10.131265 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r2mds"] Jul 06 21:22:10.134471 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:22:10.134450 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod594d6a6c_c70c_4642_9c6b_3745fbd64c4f.slice/crio-8901b178ffb12d581cf8a512b9494f35ebf9cf5b54e15faac9bc34cabd9c1aa9 WatchSource:0}: Error finding container 8901b178ffb12d581cf8a512b9494f35ebf9cf5b54e15faac9bc34cabd9c1aa9: Status 404 returned error can't find the container with id 8901b178ffb12d581cf8a512b9494f35ebf9cf5b54e15faac9bc34cabd9c1aa9 Jul 06 21:22:10.366523 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:10.366494 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:10.366878 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:10.366648 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 21:22:10.366878 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:10.366714 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls podName:96f500f5-8245-4d8b-98f4-60c5ff8821ef nodeName:}" failed. No retries permitted until 2026-07-06 21:22:11.36669605 +0000 UTC m=+39.118319092 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls") pod "dns-default-qdv89" (UID: "96f500f5-8245-4d8b-98f4-60c5ff8821ef") : secret "dns-default-metrics-tls" not found Jul 06 21:22:10.391978 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:10.391888 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r2mds" event={"ID":"594d6a6c-c70c-4642-9c6b-3745fbd64c4f","Type":"ContainerStarted","Data":"8901b178ffb12d581cf8a512b9494f35ebf9cf5b54e15faac9bc34cabd9c1aa9"} Jul 06 21:22:10.394508 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:10.394484 2573 generic.go:356] "Generic (PLEG): container finished" podID="5b843783-33a8-4332-8d96-a21b5abc3a40" containerID="d6886d385160df3b91850a8a9e80537415b8fdae3aaf2c8533994bb5c6daa039" exitCode=0 Jul 06 21:22:10.394618 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:10.394530 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsbrr" event={"ID":"5b843783-33a8-4332-8d96-a21b5abc3a40","Type":"ContainerDied","Data":"d6886d385160df3b91850a8a9e80537415b8fdae3aaf2c8533994bb5c6daa039"} Jul 06 21:22:10.418446 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:10.418417 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:10.605707 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:10.605675 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:11.212379 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:11.212350 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:22:11.212531 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:11.212350 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:22:11.212595 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:11.212350 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:22:11.215572 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:11.215550 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 21:22:11.216582 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:11.216561 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lhqbc\"" Jul 06 21:22:11.216716 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:11.216615 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 21:22:11.216879 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:11.216860 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 21:22:11.216879 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:11.216876 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m4jxb\"" Jul 06 21:22:11.217073 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:11.216878 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 21:22:11.373600 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:11.373566 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:11.374072 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:11.373688 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 21:22:11.374072 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:11.373741 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls podName:96f500f5-8245-4d8b-98f4-60c5ff8821ef nodeName:}" failed. No retries permitted until 2026-07-06 21:22:13.373727053 +0000 UTC m=+41.125350117 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls") pod "dns-default-qdv89" (UID: "96f500f5-8245-4d8b-98f4-60c5ff8821ef") : secret "dns-default-metrics-tls" not found Jul 06 21:22:11.403727 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:11.403695 2573 generic.go:356] "Generic (PLEG): container finished" podID="5b843783-33a8-4332-8d96-a21b5abc3a40" containerID="bc19b5cbd2ee36e536f58d5fcf329d88ca1c647dc45e97e53052989bd125f3ab" exitCode=0 Jul 06 21:22:11.403873 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:11.403741 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsbrr" event={"ID":"5b843783-33a8-4332-8d96-a21b5abc3a40","Type":"ContainerDied","Data":"bc19b5cbd2ee36e536f58d5fcf329d88ca1c647dc45e97e53052989bd125f3ab"} Jul 06 21:22:12.276106 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:12.276082 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:12.349572 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:12.349505 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:12.410483 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:12.410450 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vsbrr" event={"ID":"5b843783-33a8-4332-8d96-a21b5abc3a40","Type":"ContainerStarted","Data":"69f08e27f100c18f69dd3568379d6b73365378aaffc4f92286fa1154deec1b8a"} Jul 06 21:22:12.411961 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:12.411918 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r2mds" event={"ID":"594d6a6c-c70c-4642-9c6b-3745fbd64c4f","Type":"ContainerStarted","Data":"9013fbf7cc09fe4311506280bfc9c6fd8cc56a9eee381a755df72ec9721fa296"} Jul 06 21:22:12.443997 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:12.443923 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vsbrr" podStartSLOduration=8.290217809 podStartE2EDuration="38.443904222s" totalImagesPullingTime="30.153686413s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:21:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:22:12.440797694 +0000 UTC m=+40.192420757" watchObservedRunningTime="2026-07-06 21:22:12.443904222 +0000 UTC m=+40.195527287" Jul 06 21:22:12.458532 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:12.458483 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-r2mds" podStartSLOduration=1.3396055150000001 podStartE2EDuration="3.458467726s" totalImagesPullingTime="2.118862211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:22:12.457711843 +0000 UTC m=+40.209334893" watchObservedRunningTime="2026-07-06 21:22:12.458467726 +0000 UTC m=+40.210090790" Jul 06 21:22:12.577269 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:12.577236 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:12.677918 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:12.677839 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:13.082924 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:13.082881 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:13.183616 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:13.183584 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:13.374546 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:13.374519 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:13.388907 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:13.388881 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:13.389036 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:13.388993 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 21:22:13.389084 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:13.389049 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls podName:96f500f5-8245-4d8b-98f4-60c5ff8821ef nodeName:}" failed. No retries permitted until 2026-07-06 21:22:17.389035687 +0000 UTC m=+45.140658729 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls") pod "dns-default-qdv89" (UID: "96f500f5-8245-4d8b-98f4-60c5ff8821ef") : secret "dns-default-metrics-tls" not found Jul 06 21:22:13.454821 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:13.454792 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:13.545704 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:13.545679 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:13.622429 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:13.622402 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:17.413604 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:17.413566 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:17.413997 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:17.413688 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 21:22:17.413997 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:17.413745 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls podName:96f500f5-8245-4d8b-98f4-60c5ff8821ef nodeName:}" failed. No retries permitted until 2026-07-06 21:22:25.413729296 +0000 UTC m=+53.165352339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls") pod "dns-default-qdv89" (UID: "96f500f5-8245-4d8b-98f4-60c5ff8821ef") : secret "dns-default-metrics-tls" not found Jul 06 21:22:18.562736 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.562705 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:18.673505 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.673467 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:18.702692 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.702662 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8wmpm"] Jul 06 21:22:18.719409 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.719386 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:18.721122 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.721097 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8wmpm"] Jul 06 21:22:18.722232 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.722211 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 21:22:18.722232 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.722226 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 21:22:18.722462 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.722446 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 21:22:18.723379 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.723357 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-69gp7\"" Jul 06 21:22:18.723523 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.723380 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 21:22:18.747687 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.747662 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:18.822567 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.822515 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/31c8be7f-93b2-47a2-aaa2-50048aa25e93-crio-socket\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:18.822567 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.822540 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/31c8be7f-93b2-47a2-aaa2-50048aa25e93-data-volume\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:18.822694 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.822589 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:18.822694 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.822656 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:18.822694 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.822693 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f749d\" (UniqueName: \"kubernetes.io/projected/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-api-access-f749d\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:18.923363 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.923335 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:18.923472 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.923387 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f749d\" (UniqueName: \"kubernetes.io/projected/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-api-access-f749d\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:18.923510 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:18.923474 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 21:22:18.923542 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.923520 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/31c8be7f-93b2-47a2-aaa2-50048aa25e93-crio-socket\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:18.923575 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:18.923550 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls podName:31c8be7f-93b2-47a2-aaa2-50048aa25e93 nodeName:}" failed. No retries permitted until 2026-07-06 21:22:19.423531963 +0000 UTC m=+47.175155021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8wmpm" (UID: "31c8be7f-93b2-47a2-aaa2-50048aa25e93") : secret "insights-runtime-extractor-tls" not found Jul 06 21:22:18.923632 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.923577 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/31c8be7f-93b2-47a2-aaa2-50048aa25e93-data-volume\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:18.923670 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.923636 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:18.923717 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.923696 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/31c8be7f-93b2-47a2-aaa2-50048aa25e93-crio-socket\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:18.923766 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:18.923727 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 21:22:18.923813 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:18.923777 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm podName:31c8be7f-93b2-47a2-aaa2-50048aa25e93 nodeName:}" failed. No retries permitted until 2026-07-06 21:22:19.423767629 +0000 UTC m=+47.175390670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8wmpm" (UID: "31c8be7f-93b2-47a2-aaa2-50048aa25e93") : configmap "kube-rbac-proxy" not found Jul 06 21:22:18.924001 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.923982 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/31c8be7f-93b2-47a2-aaa2-50048aa25e93-data-volume\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:18.934091 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:18.934072 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f749d\" (UniqueName: \"kubernetes.io/projected/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-api-access-f749d\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:19.077081 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.077003 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:19.104092 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.104063 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-g7mzb"] Jul 06 21:22:19.122697 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.122676 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-g7mzb"] Jul 06 21:22:19.122840 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.122765 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-g7mzb" Jul 06 21:22:19.126470 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.126448 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-h5g2s\"" Jul 06 21:22:19.126911 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.126893 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 06 21:22:19.131845 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.131827 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 06 21:22:19.175848 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.175822 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:19.226209 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.226183 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sprpq\" (UniqueName: \"kubernetes.io/projected/8e23ff36-46e5-4c85-bba2-01ffa8ae2fac-kube-api-access-sprpq\") pod \"migrator-d57d58cc9-g7mzb\" (UID: \"8e23ff36-46e5-4c85-bba2-01ffa8ae2fac\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-g7mzb" Jul 06 21:22:19.311036 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.311009 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:19.327297 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.327252 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sprpq\" (UniqueName: \"kubernetes.io/projected/8e23ff36-46e5-4c85-bba2-01ffa8ae2fac-kube-api-access-sprpq\") pod \"migrator-d57d58cc9-g7mzb\" (UID: \"8e23ff36-46e5-4c85-bba2-01ffa8ae2fac\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-g7mzb" Jul 06 21:22:19.350488 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.350465 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sprpq\" (UniqueName: \"kubernetes.io/projected/8e23ff36-46e5-4c85-bba2-01ffa8ae2fac-kube-api-access-sprpq\") pod \"migrator-d57d58cc9-g7mzb\" (UID: \"8e23ff36-46e5-4c85-bba2-01ffa8ae2fac\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-g7mzb" Jul 06 21:22:19.428569 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.428545 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:19.428661 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.428585 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:19.428703 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:19.428663 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 21:22:19.428703 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:19.428683 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 21:22:19.428766 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:19.428727 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm podName:31c8be7f-93b2-47a2-aaa2-50048aa25e93 nodeName:}" failed. No retries permitted until 2026-07-06 21:22:20.428709254 +0000 UTC m=+48.180332300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8wmpm" (UID: "31c8be7f-93b2-47a2-aaa2-50048aa25e93") : configmap "kube-rbac-proxy" not found Jul 06 21:22:19.428766 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:19.428745 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls podName:31c8be7f-93b2-47a2-aaa2-50048aa25e93 nodeName:}" failed. No retries permitted until 2026-07-06 21:22:20.428737423 +0000 UTC m=+48.180360464 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8wmpm" (UID: "31c8be7f-93b2-47a2-aaa2-50048aa25e93") : secret "insights-runtime-extractor-tls" not found Jul 06 21:22:19.431256 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.431243 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-g7mzb" Jul 06 21:22:19.431637 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.431615 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:19.592596 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.592528 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-g7mzb"] Jul 06 21:22:19.595834 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:22:19.595811 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e23ff36_46e5_4c85_bba2_01ffa8ae2fac.slice/crio-2932f05f8577fa691e3a2475f8fc6b850512bd617610cc68b5af97d6a763835a WatchSource:0}: Error finding container 2932f05f8577fa691e3a2475f8fc6b850512bd617610cc68b5af97d6a763835a: Status 404 returned error can't find the container with id 2932f05f8577fa691e3a2475f8fc6b850512bd617610cc68b5af97d6a763835a Jul 06 21:22:19.847193 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.847118 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:19.947720 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:19.947690 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:20.435291 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:20.435258 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:20.435485 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:20.435434 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 21:22:20.435485 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:20.435479 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 21:22:20.435586 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:20.435434 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:20.435586 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:20.435500 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls podName:31c8be7f-93b2-47a2-aaa2-50048aa25e93 nodeName:}" failed. No retries permitted until 2026-07-06 21:22:22.435481254 +0000 UTC m=+50.187104299 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8wmpm" (UID: "31c8be7f-93b2-47a2-aaa2-50048aa25e93") : secret "insights-runtime-extractor-tls" not found Jul 06 21:22:20.435586 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:20.435537 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm podName:31c8be7f-93b2-47a2-aaa2-50048aa25e93 nodeName:}" failed. No retries permitted until 2026-07-06 21:22:22.435523671 +0000 UTC m=+50.187146713 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8wmpm" (UID: "31c8be7f-93b2-47a2-aaa2-50048aa25e93") : configmap "kube-rbac-proxy" not found Jul 06 21:22:20.445656 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:20.445629 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-g7mzb" event={"ID":"8e23ff36-46e5-4c85-bba2-01ffa8ae2fac","Type":"ContainerStarted","Data":"2932f05f8577fa691e3a2475f8fc6b850512bd617610cc68b5af97d6a763835a"} Jul 06 21:22:20.450715 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:20.450695 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:20.551709 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:20.551672 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:20.651784 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:20.651750 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:20.679201 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:20.679181 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qmqnl_e7231950-58b8-40d6-ad8d-7a57df972c1f/dns-node-resolver/0.log" Jul 06 21:22:21.453053 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:21.453014 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-g7mzb" event={"ID":"8e23ff36-46e5-4c85-bba2-01ffa8ae2fac","Type":"ContainerStarted","Data":"e06d61ce36e02b45e9f603f1864b96f7fc4775cc81d788ea4924e2901da79f21"} Jul 06 21:22:21.495824 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:21.495802 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:21.596523 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:21.596455 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:21.680155 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:21.680131 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7djjr_ee182599-bb07-4f59-bc97-1efdff4e0da8/node-ca/0.log" Jul 06 21:22:22.450547 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:22.450512 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:22.450705 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:22.450559 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:22.450705 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:22.450632 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 21:22:22.450705 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:22.450681 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm podName:31c8be7f-93b2-47a2-aaa2-50048aa25e93 nodeName:}" failed. No retries permitted until 2026-07-06 21:22:26.45066454 +0000 UTC m=+54.202287583 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8wmpm" (UID: "31c8be7f-93b2-47a2-aaa2-50048aa25e93") : configmap "kube-rbac-proxy" not found Jul 06 21:22:22.450808 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:22.450713 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 21:22:22.450808 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:22.450766 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls podName:31c8be7f-93b2-47a2-aaa2-50048aa25e93 nodeName:}" failed. No retries permitted until 2026-07-06 21:22:26.450748459 +0000 UTC m=+54.202371502 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8wmpm" (UID: "31c8be7f-93b2-47a2-aaa2-50048aa25e93") : secret "insights-runtime-extractor-tls" not found Jul 06 21:22:22.456845 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:22.456817 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-g7mzb" event={"ID":"8e23ff36-46e5-4c85-bba2-01ffa8ae2fac","Type":"ContainerStarted","Data":"fa486af3298a8bff60bd5a3bee835fceae47c9f4f343eff52fcddd7f25b4218b"} Jul 06 21:22:22.519744 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:22.519714 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:22.680317 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:22.680291 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r2mds_594d6a6c-c70c-4642-9c6b-3745fbd64c4f/serve-healthcheck-canary/0.log" Jul 06 21:22:23.910208 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:23.910170 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:25.474792 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:25.474763 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:25.475206 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:25.474859 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 21:22:25.475206 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:25.474907 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls podName:96f500f5-8245-4d8b-98f4-60c5ff8821ef nodeName:}" failed. No retries permitted until 2026-07-06 21:22:41.474893989 +0000 UTC m=+69.226517031 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls") pod "dns-default-qdv89" (UID: "96f500f5-8245-4d8b-98f4-60c5ff8821ef") : secret "dns-default-metrics-tls" not found Jul 06 21:22:25.795061 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:25.795033 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:26.077873 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:26.077793 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:26.181889 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:26.181854 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:26.381971 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:26.381870 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:26.479687 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:26.479658 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:26.480057 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:26.479669 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:26.480057 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:26.479774 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:26.480057 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:26.479887 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 21:22:26.480057 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:22:26.479969 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls podName:31c8be7f-93b2-47a2-aaa2-50048aa25e93 nodeName:}" failed. No retries permitted until 2026-07-06 21:22:34.479932497 +0000 UTC m=+62.231555553 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8wmpm" (UID: "31c8be7f-93b2-47a2-aaa2-50048aa25e93") : secret "insights-runtime-extractor-tls" not found Jul 06 21:22:26.480190 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:26.480076 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31c8be7f-93b2-47a2-aaa2-50048aa25e93-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:26.863764 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:26.863731 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:27.126078 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:27.125996 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:28.060325 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:28.060296 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:28.288429 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:28.288396 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:28.468411 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:28.468331 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:28.675355 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:28.675323 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:28.853625 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:28.853598 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:29.484337 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:29.484303 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:29.559813 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:29.559780 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:29.854295 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:29.854262 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:30.287752 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:30.287720 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:30.367141 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:30.367111 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:30.449104 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:30.449077 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:30.647281 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:30.647203 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:31.051464 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:31.051435 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:31.861080 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:31.861047 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:32.032330 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:32.032298 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:32.104906 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:32.104867 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:32.445925 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:32.445894 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:32.645500 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:32.645472 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:33.054008 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:33.053982 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:33.507437 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:33.507359 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:33.651139 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:33.651106 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:34.257715 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:34.257663 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:34.452490 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:34.452450 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:34.538143 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:34.538114 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:34.540349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:34.540327 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31c8be7f-93b2-47a2-aaa2-50048aa25e93-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8wmpm\" (UID: \"31c8be7f-93b2-47a2-aaa2-50048aa25e93\") " pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:34.632225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:34.632195 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-69gp7\"" Jul 06 21:22:34.639465 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:34.639446 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8wmpm" Jul 06 21:22:34.793968 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:34.793858 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-g7mzb" podStartSLOduration=14.053551348 podStartE2EDuration="15.793837681s" totalImagesPullingTime="1.740286333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:22:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:22:22.483904268 +0000 UTC m=+50.235527345" watchObservedRunningTime="2026-07-06 21:22:34.793837681 +0000 UTC m=+62.545460745" Jul 06 21:22:34.794492 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:34.794475 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8wmpm"] Jul 06 21:22:34.797602 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:22:34.797570 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c8be7f_93b2_47a2_aaa2_50048aa25e93.slice/crio-1e6950f0b3e4f6def1bfa3f17e60356d9a04e79f56d9631fbdcda5c89742e333 WatchSource:0}: Error finding container 1e6950f0b3e4f6def1bfa3f17e60356d9a04e79f56d9631fbdcda5c89742e333: Status 404 returned error can't find the container with id 1e6950f0b3e4f6def1bfa3f17e60356d9a04e79f56d9631fbdcda5c89742e333 Jul 06 21:22:34.862911 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:34.862887 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:35.055759 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:35.055677 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:35.398479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:35.398409 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfjvh" Jul 06 21:22:35.490085 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:35.490047 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8wmpm" event={"ID":"31c8be7f-93b2-47a2-aaa2-50048aa25e93","Type":"ContainerStarted","Data":"1e6950f0b3e4f6def1bfa3f17e60356d9a04e79f56d9631fbdcda5c89742e333"} Jul 06 21:22:35.498199 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:35.498163 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:35.594182 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:35.594145 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:35.697816 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:35.697699 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:36.057161 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:36.057126 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:36.244653 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:36.244623 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:36.461043 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:36.460928 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:39.504898 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.504808 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8wmpm" event={"ID":"31c8be7f-93b2-47a2-aaa2-50048aa25e93","Type":"ContainerStarted","Data":"78870d07fe33d1b9817fd8dcc4f2ade8f3649f4b627e2cf3f8741e9d67e16adf"} Jul 06 21:22:39.504898 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.504887 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8wmpm" event={"ID":"31c8be7f-93b2-47a2-aaa2-50048aa25e93","Type":"ContainerStarted","Data":"95edc790c88ec5e099aeeab43defeece1969354f224ecda00bb80370fe39ea99"} Jul 06 21:22:39.680384 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.680344 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs\") pod \"network-metrics-daemon-b8dk6\" (UID: \"626f5874-a75d-42b5-b585-a019b92b5283\") " pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:22:39.680542 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.680420 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:22:39.685794 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.685770 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 21:22:39.685794 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.685789 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 21:22:39.692999 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.692965 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae8aa509-49e8-451b-8987-efcd4d6a4f1f-original-pull-secret\") pod \"global-pull-secret-syncer-zb6q8\" (UID: \"ae8aa509-49e8-451b-8987-efcd4d6a4f1f\") " pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:22:39.693138 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.693118 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/626f5874-a75d-42b5-b585-a019b92b5283-metrics-certs\") pod \"network-metrics-daemon-b8dk6\" (UID: \"626f5874-a75d-42b5-b585-a019b92b5283\") " pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:22:39.725190 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.725160 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zb6q8" Jul 06 21:22:39.741245 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.741225 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m4jxb\"" Jul 06 21:22:39.748632 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.748614 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8dk6" Jul 06 21:22:39.781340 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.781177 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5cc6\" (UniqueName: \"kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6\") pod \"network-check-target-zx6nz\" (UID: \"1f0cad70-ce55-4daf-95bf-43060c28dcae\") " pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:22:39.787284 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.787255 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 21:22:39.794902 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.794876 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 21:22:39.806130 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.806083 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5cc6\" (UniqueName: \"kubernetes.io/projected/1f0cad70-ce55-4daf-95bf-43060c28dcae-kube-api-access-s5cc6\") pod \"network-check-target-zx6nz\" (UID: \"1f0cad70-ce55-4daf-95bf-43060c28dcae\") " pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:22:39.882695 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.882655 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zb6q8"] Jul 06 21:22:39.888203 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:22:39.888162 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae8aa509_49e8_451b_8987_efcd4d6a4f1f.slice/crio-581590614c39bf9e2b728cf0d23ee631d70fe76f63c9c1f7585ea5f8f901567f WatchSource:0}: Error finding container 581590614c39bf9e2b728cf0d23ee631d70fe76f63c9c1f7585ea5f8f901567f: Status 404 returned error can't find the container with id 581590614c39bf9e2b728cf0d23ee631d70fe76f63c9c1f7585ea5f8f901567f Jul 06 21:22:39.910736 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:39.910709 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b8dk6"] Jul 06 21:22:39.914149 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:22:39.914119 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod626f5874_a75d_42b5_b585_a019b92b5283.slice/crio-14adb0c37fedc94421b060bc054ee49605411a4f6d8f4401e61bdb9a570172eb WatchSource:0}: Error finding container 14adb0c37fedc94421b060bc054ee49605411a4f6d8f4401e61bdb9a570172eb: Status 404 returned error can't find the container with id 14adb0c37fedc94421b060bc054ee49605411a4f6d8f4401e61bdb9a570172eb Jul 06 21:22:40.035292 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:40.035272 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lhqbc\"" Jul 06 21:22:40.043742 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:40.043724 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:22:40.182940 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:40.182912 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zx6nz"] Jul 06 21:22:40.186423 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:22:40.186398 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f0cad70_ce55_4daf_95bf_43060c28dcae.slice/crio-dab471efceae850a941ce8a2efec7ac8d18cb9ff3c157b3a7d659042f6183bb3 WatchSource:0}: Error finding container dab471efceae850a941ce8a2efec7ac8d18cb9ff3c157b3a7d659042f6183bb3: Status 404 returned error can't find the container with id dab471efceae850a941ce8a2efec7ac8d18cb9ff3c157b3a7d659042f6183bb3 Jul 06 21:22:40.510197 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:40.510093 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zb6q8" event={"ID":"ae8aa509-49e8-451b-8987-efcd4d6a4f1f","Type":"ContainerStarted","Data":"581590614c39bf9e2b728cf0d23ee631d70fe76f63c9c1f7585ea5f8f901567f"} Jul 06 21:22:40.511980 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:40.511931 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zx6nz" event={"ID":"1f0cad70-ce55-4daf-95bf-43060c28dcae","Type":"ContainerStarted","Data":"dab471efceae850a941ce8a2efec7ac8d18cb9ff3c157b3a7d659042f6183bb3"} Jul 06 21:22:40.513355 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:40.513328 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b8dk6" event={"ID":"626f5874-a75d-42b5-b585-a019b92b5283","Type":"ContainerStarted","Data":"14adb0c37fedc94421b060bc054ee49605411a4f6d8f4401e61bdb9a570172eb"} Jul 06 21:22:41.495540 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:41.495503 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:41.501442 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:41.501414 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96f500f5-8245-4d8b-98f4-60c5ff8821ef-metrics-tls\") pod \"dns-default-qdv89\" (UID: \"96f500f5-8245-4d8b-98f4-60c5ff8821ef\") " pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:41.597533 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:41.597497 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:41.686281 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:41.686247 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:41.716652 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:41.716620 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9fg2d\"" Jul 06 21:22:41.723757 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:41.723734 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:41.867102 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:41.867022 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:42.099957 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:42.099906 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:42.235143 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:42.235099 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:42.317824 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:42.317785 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:43.230139 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:43.230104 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:43.296453 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:43.296416 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:43.430382 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:43.430347 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:43.851854 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:43.851829 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:45.098432 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.098396 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:45.223284 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.223249 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f6f8666-kz4jm"] Jul 06 21:22:45.226101 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.226078 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f6f8666-kz4jm" Jul 06 21:22:45.229185 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.229141 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx"] Jul 06 21:22:45.229336 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.229309 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 21:22:45.230044 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.230008 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 21:22:45.230324 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.230306 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 21:22:45.230324 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.230318 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 21:22:45.230430 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.230322 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-hf5nt\"" Jul 06 21:22:45.231309 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.231254 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.234606 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.234580 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 06 21:22:45.235572 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.235544 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:45.236176 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.236160 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 06 21:22:45.236354 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.236332 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 06 21:22:45.242298 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.242276 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 06 21:22:45.242395 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.242282 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f6f8666-kz4jm"] Jul 06 21:22:45.259375 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.259350 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx"] Jul 06 21:22:45.320762 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.320732 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-hub\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.320970 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.320786 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j5cb\" (UniqueName: \"kubernetes.io/projected/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-kube-api-access-6j5cb\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.320970 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.320814 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/674f1a4e-7626-4d61-a4ef-68361649b625-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57f6f8666-kz4jm\" (UID: \"674f1a4e-7626-4d61-a4ef-68361649b625\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f6f8666-kz4jm" Jul 06 21:22:45.320970 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.320841 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-ca\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.320970 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.320860 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.320970 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.320901 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-additional-service-ca\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.321192 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.321009 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gt9b\" (UniqueName: \"kubernetes.io/projected/674f1a4e-7626-4d61-a4ef-68361649b625-kube-api-access-8gt9b\") pod \"managed-serviceaccount-addon-agent-57f6f8666-kz4jm\" (UID: \"674f1a4e-7626-4d61-a4ef-68361649b625\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f6f8666-kz4jm" Jul 06 21:22:45.321192 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.321053 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.360237 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.360114 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:45.422145 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.422117 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.422257 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.422159 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-hub\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.422257 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.422203 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j5cb\" (UniqueName: \"kubernetes.io/projected/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-kube-api-access-6j5cb\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.422257 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.422232 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/674f1a4e-7626-4d61-a4ef-68361649b625-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57f6f8666-kz4jm\" (UID: \"674f1a4e-7626-4d61-a4ef-68361649b625\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f6f8666-kz4jm" Jul 06 21:22:45.422430 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.422259 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-ca\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.422430 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.422285 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.422430 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.422327 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-additional-service-ca\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.422430 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.422361 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gt9b\" (UniqueName: \"kubernetes.io/projected/674f1a4e-7626-4d61-a4ef-68361649b625-kube-api-access-8gt9b\") pod \"managed-serviceaccount-addon-agent-57f6f8666-kz4jm\" (UID: \"674f1a4e-7626-4d61-a4ef-68361649b625\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f6f8666-kz4jm" Jul 06 21:22:45.424212 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.423684 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-additional-service-ca\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.426004 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.425580 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.426004 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.425914 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-ca\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.426173 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.426015 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.426173 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.426119 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/674f1a4e-7626-4d61-a4ef-68361649b625-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57f6f8666-kz4jm\" (UID: \"674f1a4e-7626-4d61-a4ef-68361649b625\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f6f8666-kz4jm" Jul 06 21:22:45.426328 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.426171 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-hub\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.441972 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.441610 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gt9b\" (UniqueName: \"kubernetes.io/projected/674f1a4e-7626-4d61-a4ef-68361649b625-kube-api-access-8gt9b\") pod \"managed-serviceaccount-addon-agent-57f6f8666-kz4jm\" (UID: \"674f1a4e-7626-4d61-a4ef-68361649b625\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f6f8666-kz4jm" Jul 06 21:22:45.441972 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.441906 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j5cb\" (UniqueName: \"kubernetes.io/projected/ea1f6c5e-ee04-46c7-b7e7-a61286253bee-kube-api-access-6j5cb\") pod \"cluster-proxy-proxy-agent-657f49cc7d-x2tkx\" (UID: \"ea1f6c5e-ee04-46c7-b7e7-a61286253bee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.485774 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.485695 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:45.541741 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.541707 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8wmpm" event={"ID":"31c8be7f-93b2-47a2-aaa2-50048aa25e93","Type":"ContainerStarted","Data":"cf2e6a5d4d8d00eeff1643ea5186c9d0a75d74ea6697e0fb8de6866782834ea8"} Jul 06 21:22:45.545971 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.545648 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f6f8666-kz4jm" Jul 06 21:22:45.552448 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.552419 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" Jul 06 21:22:45.579833 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.579780 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8wmpm" podStartSLOduration=17.056106328 podStartE2EDuration="27.579760091s" totalImagesPullingTime="10.523653763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:22:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:22:45.579718339 +0000 UTC m=+73.331341404" watchObservedRunningTime="2026-07-06 21:22:45.579760091 +0000 UTC m=+73.331383156" Jul 06 21:22:45.586205 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.585402 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qdv89"] Jul 06 21:22:45.626107 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.625824 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:45.717391 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.717122 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f6f8666-kz4jm"] Jul 06 21:22:45.721208 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:22:45.721180 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod674f1a4e_7626_4d61_a4ef_68361649b625.slice/crio-07ac8d3c915f305ab4190ef5bbc4cad19f5c27b244c997a2465a8066f19fb7fc WatchSource:0}: Error finding container 07ac8d3c915f305ab4190ef5bbc4cad19f5c27b244c997a2465a8066f19fb7fc: Status 404 returned error can't find the container with id 07ac8d3c915f305ab4190ef5bbc4cad19f5c27b244c997a2465a8066f19fb7fc Jul 06 21:22:45.727871 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.727837 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:45.736564 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.736540 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx"] Jul 06 21:22:45.742693 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:22:45.742671 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea1f6c5e_ee04_46c7_b7e7_a61286253bee.slice/crio-7b125dd3e47bd8d137069a9ea70a768a4b4dea8c20b48358544eea0030614332 WatchSource:0}: Error finding container 7b125dd3e47bd8d137069a9ea70a768a4b4dea8c20b48358544eea0030614332: Status 404 returned error can't find the container with id 7b125dd3e47bd8d137069a9ea70a768a4b4dea8c20b48358544eea0030614332 Jul 06 21:22:45.846593 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:45.846558 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:46.049446 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.049406 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:46.169196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.169164 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:46.268193 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.268113 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:46.558775 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.553080 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qdv89" event={"ID":"96f500f5-8245-4d8b-98f4-60c5ff8821ef","Type":"ContainerStarted","Data":"8353de36714611d7fa3709ba3c8cf36f31ca651679b4c4f1aa28abc4ea0253e3"} Jul 06 21:22:46.558775 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.556863 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zx6nz" event={"ID":"1f0cad70-ce55-4daf-95bf-43060c28dcae","Type":"ContainerStarted","Data":"0f9ed14ca3316822e4f013d4a941e12245f11f0dec0d34a464f61230fff5e2d1"} Jul 06 21:22:46.558775 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.557635 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:22:46.562617 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.562590 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b8dk6" event={"ID":"626f5874-a75d-42b5-b585-a019b92b5283","Type":"ContainerStarted","Data":"cb832f51ce4773c1815e40a50a32c0268d6941baae0f5a6852da6178a548baa2"} Jul 06 21:22:46.562739 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.562643 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b8dk6" event={"ID":"626f5874-a75d-42b5-b585-a019b92b5283","Type":"ContainerStarted","Data":"b8c01a696e961217f36e3b41e54a4c1935a3ffa77cd100c38e0b6d36a96c1427"} Jul 06 21:22:46.566387 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.566343 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f6f8666-kz4jm" event={"ID":"674f1a4e-7626-4d61-a4ef-68361649b625","Type":"ContainerStarted","Data":"07ac8d3c915f305ab4190ef5bbc4cad19f5c27b244c997a2465a8066f19fb7fc"} Jul 06 21:22:46.572427 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.572402 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zb6q8" event={"ID":"ae8aa509-49e8-451b-8987-efcd4d6a4f1f","Type":"ContainerStarted","Data":"fc0cb95341d1a3167f03223dc4d0a0897f175ea5e9439d7385074b2bfd14a571"} Jul 06 21:22:46.574690 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.574664 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" event={"ID":"ea1f6c5e-ee04-46c7-b7e7-a61286253bee","Type":"ContainerStarted","Data":"7b125dd3e47bd8d137069a9ea70a768a4b4dea8c20b48358544eea0030614332"} Jul 06 21:22:46.609555 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.609498 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zx6nz" podStartSLOduration=67.359376817 podStartE2EDuration="1m12.609476569s" totalImagesPullingTime="5.250099752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:22:46.587369095 +0000 UTC m=+74.338992161" watchObservedRunningTime="2026-07-06 21:22:46.609476569 +0000 UTC m=+74.361099634" Jul 06 21:22:46.639833 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.638934 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zb6q8" podStartSLOduration=67.071841019 podStartE2EDuration="1m12.638914554s" totalImagesPullingTime="5.567073535s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:22:46.610400719 +0000 UTC m=+74.362023778" watchObservedRunningTime="2026-07-06 21:22:46.638914554 +0000 UTC m=+74.390537619" Jul 06 21:22:46.639833 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.639468 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b8dk6" podStartSLOduration=67.140915121 podStartE2EDuration="1m12.639456501s" totalImagesPullingTime="5.49854138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:22:46.638783058 +0000 UTC m=+74.390406125" watchObservedRunningTime="2026-07-06 21:22:46.639456501 +0000 UTC m=+74.391079563" Jul 06 21:22:46.744791 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.744692 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:46.817398 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:46.817289 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:47.934340 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:47.934293 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:48.186245 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:48.186155 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:48.266514 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:48.266477 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:49.269768 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:49.269733 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:49.345704 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:49.345673 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:50.522097 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:50.522062 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:50.591197 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:50.591157 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" event={"ID":"ea1f6c5e-ee04-46c7-b7e7-a61286253bee","Type":"ContainerStarted","Data":"974ccbefd1384dba63b65da5d95ba82c2ecc903b0bd28eb8acd06ef8dbdf3edf"} Jul 06 21:22:50.591360 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:50.591212 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" event={"ID":"ea1f6c5e-ee04-46c7-b7e7-a61286253bee","Type":"ContainerStarted","Data":"e39e180f107ab879f33c1c0b69535e82469c590232afa87ba352762e2255814b"} Jul 06 21:22:50.591360 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:50.591227 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" event={"ID":"ea1f6c5e-ee04-46c7-b7e7-a61286253bee","Type":"ContainerStarted","Data":"8d95a4babd8924b103630be94e38b5b196e100f6e6be7b0d9bd36108ed273603"} Jul 06 21:22:50.592998 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:50.592972 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qdv89" event={"ID":"96f500f5-8245-4d8b-98f4-60c5ff8821ef","Type":"ContainerStarted","Data":"6a454defb7d654b13b10407f7f046ecf1a8bea6a14861d036240bae02c72e4ec"} Jul 06 21:22:50.593119 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:50.593021 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qdv89" event={"ID":"96f500f5-8245-4d8b-98f4-60c5ff8821ef","Type":"ContainerStarted","Data":"fbe2459efaa7c0fbdf4e3903401928516314cc71866e9c5ece5fe4df2729f881"} Jul 06 21:22:50.593119 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:50.593050 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qdv89" Jul 06 21:22:50.594238 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:50.594213 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f6f8666-kz4jm" event={"ID":"674f1a4e-7626-4d61-a4ef-68361649b625","Type":"ContainerStarted","Data":"6d6fd407915b6af06d3366ea459792acfa31e7dfa1d497e70bd958c5b3458e88"} Jul 06 21:22:50.615474 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:50.615429 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-657f49cc7d-x2tkx" podStartSLOduration=1.5281693189999999 podStartE2EDuration="5.615415902s" totalImagesPullingTime="4.087246583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:22:50.613848387 +0000 UTC m=+78.365471481" watchObservedRunningTime="2026-07-06 21:22:50.615415902 +0000 UTC m=+78.367038966" Jul 06 21:22:50.619387 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:50.619367 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:50.651825 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:50.651778 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57f6f8666-kz4jm" podStartSLOduration=1.55458346 podStartE2EDuration="5.651763982s" totalImagesPullingTime="4.097180522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:22:50.631845384 +0000 UTC m=+78.383468447" watchObservedRunningTime="2026-07-06 21:22:50.651763982 +0000 UTC m=+78.403387064" Jul 06 21:22:50.652128 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:50.652086 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qdv89" podStartSLOduration=37.440953583 podStartE2EDuration="41.652075143s" totalImagesPullingTime="4.21112156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:22:50.651823772 +0000 UTC m=+78.403446852" watchObservedRunningTime="2026-07-06 21:22:50.652075143 +0000 UTC m=+78.403698207" Jul 06 21:22:51.144998 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:51.144960 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:51.212276 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:51.212249 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:51.452618 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:51.452534 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:51.527290 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:51.527255 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:53.788282 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:53.788250 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:56.259520 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.259485 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:56.477623 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.477582 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:56.619560 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.619528 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mmphz"] Jul 06 21:22:56.624069 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.624054 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.655852 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.655830 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 21:22:56.656510 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.656496 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zzc4v\"" Jul 06 21:22:56.656795 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.656777 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 21:22:56.664500 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.664486 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 21:22:56.664554 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.664533 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 21:22:56.670075 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.670059 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 21:22:56.670174 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.670161 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 21:22:56.692678 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.692657 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:56.698730 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.698709 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bce37c5-e997-4fa3-b132-e15fc9321a35-sys\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.698795 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.698738 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-textfile\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.698795 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.698758 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-wtmp\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.698795 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.698775 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bce37c5-e997-4fa3-b132-e15fc9321a35-metrics-client-ca\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.698933 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.698810 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7bce37c5-e997-4fa3-b132-e15fc9321a35-root\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.698933 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.698829 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-accelerators-collector-config\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.698933 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.698845 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4sxj\" (UniqueName: \"kubernetes.io/projected/7bce37c5-e997-4fa3-b132-e15fc9321a35-kube-api-access-f4sxj\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.698933 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.698871 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-tls\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.698933 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.698907 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800182 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800147 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7bce37c5-e997-4fa3-b132-e15fc9321a35-root\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800182 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800180 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-accelerators-collector-config\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800379 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800197 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4sxj\" (UniqueName: \"kubernetes.io/projected/7bce37c5-e997-4fa3-b132-e15fc9321a35-kube-api-access-f4sxj\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800379 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800219 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-tls\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800379 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800240 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800379 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800257 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7bce37c5-e997-4fa3-b132-e15fc9321a35-root\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800379 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800266 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bce37c5-e997-4fa3-b132-e15fc9321a35-sys\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800379 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800312 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bce37c5-e997-4fa3-b132-e15fc9321a35-sys\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800379 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800337 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-textfile\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800708 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800383 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-wtmp\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800708 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800416 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bce37c5-e997-4fa3-b132-e15fc9321a35-metrics-client-ca\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800708 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800572 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-wtmp\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800804 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800705 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-textfile\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800844 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800799 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-accelerators-collector-config\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.800920 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.800901 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bce37c5-e997-4fa3-b132-e15fc9321a35-metrics-client-ca\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.802651 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.802633 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.802698 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.802648 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7bce37c5-e997-4fa3-b132-e15fc9321a35-node-exporter-tls\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.815608 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.815581 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4sxj\" (UniqueName: \"kubernetes.io/projected/7bce37c5-e997-4fa3-b132-e15fc9321a35-kube-api-access-f4sxj\") pod \"node-exporter-mmphz\" (UID: \"7bce37c5-e997-4fa3-b132-e15fc9321a35\") " pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.852798 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.852776 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:56.932508 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.932460 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mmphz" Jul 06 21:22:56.946178 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:22:56.946148 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bce37c5_e997_4fa3_b132_e15fc9321a35.slice/crio-0e611023b218b80a180cda36521cf6c09697aa21abfbf25b568121417b85909e WatchSource:0}: Error finding container 0e611023b218b80a180cda36521cf6c09697aa21abfbf25b568121417b85909e: Status 404 returned error can't find the container with id 0e611023b218b80a180cda36521cf6c09697aa21abfbf25b568121417b85909e Jul 06 21:22:56.947934 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:56.947909 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:57.123271 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:57.123237 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:57.232294 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:57.232218 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:57.454709 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:57.454681 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:57.540452 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:57.540418 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:57.618828 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:57.618793 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mmphz" event={"ID":"7bce37c5-e997-4fa3-b132-e15fc9321a35","Type":"ContainerStarted","Data":"0e611023b218b80a180cda36521cf6c09697aa21abfbf25b568121417b85909e"} Jul 06 21:22:57.796322 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:57.796244 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:57.997235 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:57.997205 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:58.094801 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:58.094717 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:58.623394 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:58.623365 2573 generic.go:356] "Generic (PLEG): container finished" podID="7bce37c5-e997-4fa3-b132-e15fc9321a35" containerID="c178aac023fce54f21099351a34daf02e0d29666a0d5f57231cb2307a274dc02" exitCode=0 Jul 06 21:22:58.623776 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:58.623420 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mmphz" event={"ID":"7bce37c5-e997-4fa3-b132-e15fc9321a35","Type":"ContainerDied","Data":"c178aac023fce54f21099351a34daf02e0d29666a0d5f57231cb2307a274dc02"} Jul 06 21:22:59.629576 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:59.629540 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mmphz" event={"ID":"7bce37c5-e997-4fa3-b132-e15fc9321a35","Type":"ContainerStarted","Data":"dc86a51fe79904de4d54b77456b8be99e6e60dd0a5a7647aaa7b44aebc43b1be"} Jul 06 21:22:59.629997 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:59.629626 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mmphz" event={"ID":"7bce37c5-e997-4fa3-b132-e15fc9321a35","Type":"ContainerStarted","Data":"e04f6b9d2f48c6ea8d63324a84765349a2111f0e34d35f7be4e1757c0b9c2809"} Jul 06 21:22:59.684523 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:59.684481 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:59.781690 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:59.781659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:22:59.929385 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:22:59.929298 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:00.094401 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:00.094363 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:00.455615 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:00.455583 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:00.599581 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:00.599554 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qdv89" Jul 06 21:23:00.638798 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:00.638753 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mmphz" podStartSLOduration=3.6584411169999997 podStartE2EDuration="4.638739204s" totalImagesPullingTime="919.208489ms" totalInitContainerRuntime="61.089598ms" isStatefulPod=true podCreationTimestamp="2026-07-06 21:22:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:22:59.65573764 +0000 UTC m=+87.407360706" watchObservedRunningTime="2026-07-06 21:23:00.638739204 +0000 UTC m=+88.390362283" Jul 06 21:23:00.771570 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:00.771497 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:00.872772 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:00.872749 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:01.275870 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.275838 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:01.287674 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.287649 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59"] Jul 06 21:23:01.290652 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.290637 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59" Jul 06 21:23:01.292981 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.292939 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 21:23:01.293071 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.293012 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-7p94p\"" Jul 06 21:23:01.303016 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.302997 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59"] Jul 06 21:23:01.350173 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.350146 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:01.433358 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.433321 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm7gt\" (UniqueName: \"kubernetes.io/projected/64ee0da1-b6ec-40eb-880e-0a4b436f6df6-kube-api-access-dm7gt\") pod \"monitoring-plugin-d5dcff7cf-pdm59\" (UID: \"64ee0da1-b6ec-40eb-880e-0a4b436f6df6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59" Jul 06 21:23:01.433517 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.433384 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/64ee0da1-b6ec-40eb-880e-0a4b436f6df6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-pdm59\" (UID: \"64ee0da1-b6ec-40eb-880e-0a4b436f6df6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59" Jul 06 21:23:01.467522 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.467499 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:01.534238 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.534211 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/64ee0da1-b6ec-40eb-880e-0a4b436f6df6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-pdm59\" (UID: \"64ee0da1-b6ec-40eb-880e-0a4b436f6df6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59" Jul 06 21:23:01.534371 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.534257 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm7gt\" (UniqueName: \"kubernetes.io/projected/64ee0da1-b6ec-40eb-880e-0a4b436f6df6-kube-api-access-dm7gt\") pod \"monitoring-plugin-d5dcff7cf-pdm59\" (UID: \"64ee0da1-b6ec-40eb-880e-0a4b436f6df6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59" Jul 06 21:23:01.536636 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.536614 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/64ee0da1-b6ec-40eb-880e-0a4b436f6df6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-pdm59\" (UID: \"64ee0da1-b6ec-40eb-880e-0a4b436f6df6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59" Jul 06 21:23:01.545082 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.545052 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm7gt\" (UniqueName: \"kubernetes.io/projected/64ee0da1-b6ec-40eb-880e-0a4b436f6df6-kube-api-access-dm7gt\") pod \"monitoring-plugin-d5dcff7cf-pdm59\" (UID: \"64ee0da1-b6ec-40eb-880e-0a4b436f6df6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59" Jul 06 21:23:01.600123 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.600097 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59" Jul 06 21:23:01.747241 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:01.747212 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59"] Jul 06 21:23:01.750314 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:23:01.750290 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64ee0da1_b6ec_40eb_880e_0a4b436f6df6.slice/crio-7583ff4b079beecc0d6df3713e33e69043939f08b9eb1b94b66d04600190a6f2 WatchSource:0}: Error finding container 7583ff4b079beecc0d6df3713e33e69043939f08b9eb1b94b66d04600190a6f2: Status 404 returned error can't find the container with id 7583ff4b079beecc0d6df3713e33e69043939f08b9eb1b94b66d04600190a6f2 Jul 06 21:23:02.653098 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.653057 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59" event={"ID":"64ee0da1-b6ec-40eb-880e-0a4b436f6df6","Type":"ContainerStarted","Data":"7583ff4b079beecc0d6df3713e33e69043939f08b9eb1b94b66d04600190a6f2"} Jul 06 21:23:02.964476 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.964401 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 21:23:02.969503 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.969476 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:02.969680 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.969604 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:02.972547 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.972517 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 21:23:02.973156 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.973134 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 21:23:02.973377 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.973190 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4q8v00j0h5bji\"" Jul 06 21:23:02.973528 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.973454 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 21:23:02.973528 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.973466 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 21:23:02.973854 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.973837 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 21:23:02.974211 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.974187 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 21:23:02.974211 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.974201 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 21:23:02.974375 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.974207 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 21:23:02.975267 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.974834 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 21:23:02.975267 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.974858 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 21:23:02.975267 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.974878 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 21:23:02.975267 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.974837 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 21:23:02.975267 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.975050 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 21:23:02.975267 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.975086 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 21:23:02.975267 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.975177 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pxbln\"" Jul 06 21:23:02.977263 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.977244 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 21:23:02.992700 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:02.992677 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 21:23:03.053212 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053182 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053212 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053214 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053432 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053231 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053432 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053300 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053432 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053367 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053432 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053386 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053432 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053402 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053432 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053420 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053437 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053452 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053470 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053485 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053519 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8zq7\" (UniqueName: \"kubernetes.io/projected/4931891e-2c58-4301-85b8-86a9850e5500-kube-api-access-b8zq7\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053545 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053559 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053583 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053600 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-config\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053614 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4931891e-2c58-4301-85b8-86a9850e5500-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053631 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4931891e-2c58-4301-85b8-86a9850e5500-config-out\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.053959 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.053651 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-web-config\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.142701 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.142677 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:03.154220 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.154179 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.154336 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.154231 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.154336 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.154247 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.154336 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.154270 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.154336 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.154291 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.154336 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.154316 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.154590 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.154437 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.154590 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.154488 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.154688 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.154628 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.154688 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.154665 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8zq7\" (UniqueName: \"kubernetes.io/projected/4931891e-2c58-4301-85b8-86a9850e5500-kube-api-access-b8zq7\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.154932 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.154894 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.155060 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.154999 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.155060 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.155027 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.155060 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.155059 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.155224 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.155078 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-config\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.155224 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.155096 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4931891e-2c58-4301-85b8-86a9850e5500-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.155224 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.155118 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4931891e-2c58-4301-85b8-86a9850e5500-config-out\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.155224 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.155140 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-web-config\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.155224 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.155158 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.155224 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.155179 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.155224 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.155205 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.155569 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.155485 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.156066 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.155785 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.159269 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.159241 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.159378 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.159251 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.159777 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.159745 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.160722 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.160406 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.160722 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.160449 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4931891e-2c58-4301-85b8-86a9850e5500-config-out\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.160722 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.160600 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.160722 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.160637 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4931891e-2c58-4301-85b8-86a9850e5500-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.160722 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.160686 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.161053 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.160766 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.161117 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.161063 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.161205 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.161175 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.161390 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.161366 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.161576 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.161557 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.162840 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.162812 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-config\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.162938 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.162885 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-web-config\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.163213 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.163195 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.168225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.168203 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8zq7\" (UniqueName: \"kubernetes.io/projected/4931891e-2c58-4301-85b8-86a9850e5500-kube-api-access-b8zq7\") pod \"prometheus-k8s-0\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.281316 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.281290 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:03.338862 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.338816 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:03.419813 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.419788 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 21:23:03.421207 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:23:03.421181 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4931891e_2c58_4301_85b8_86a9850e5500.slice/crio-867af4d80cde9d0fcf2920ed43da436422558434f0811b3b3ed2a5065409fe39 WatchSource:0}: Error finding container 867af4d80cde9d0fcf2920ed43da436422558434f0811b3b3ed2a5065409fe39: Status 404 returned error can't find the container with id 867af4d80cde9d0fcf2920ed43da436422558434f0811b3b3ed2a5065409fe39 Jul 06 21:23:03.658414 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.658329 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59" event={"ID":"64ee0da1-b6ec-40eb-880e-0a4b436f6df6","Type":"ContainerStarted","Data":"fab2f6967635624af0863b69dc4d2811227837a7211217a6bcbd2b612f029367"} Jul 06 21:23:03.658580 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.658535 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59" Jul 06 21:23:03.659741 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.659708 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerStarted","Data":"867af4d80cde9d0fcf2920ed43da436422558434f0811b3b3ed2a5065409fe39"} Jul 06 21:23:03.663645 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.663628 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59" Jul 06 21:23:03.677757 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:03.677718 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-pdm59" podStartSLOduration=1.337691107 podStartE2EDuration="2.677702983s" totalImagesPullingTime="1.340011876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:23:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:23:03.676082821 +0000 UTC m=+91.427705885" watchObservedRunningTime="2026-07-06 21:23:03.677702983 +0000 UTC m=+91.429326048" Jul 06 21:23:04.665273 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:04.665240 2573 generic.go:356] "Generic (PLEG): container finished" podID="4931891e-2c58-4301-85b8-86a9850e5500" containerID="58a0a3061940bb5bd7c3044b493755ae7fc95ae2f4b3835138f8d5b246493ae4" exitCode=0 Jul 06 21:23:04.665733 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:04.665320 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerDied","Data":"58a0a3061940bb5bd7c3044b493755ae7fc95ae2f4b3835138f8d5b246493ae4"} Jul 06 21:23:05.540237 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:05.540198 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:05.664294 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:05.664262 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:05.766467 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:05.766439 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:07.846540 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:07.846509 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:07.920479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:07.920405 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:08.683265 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:08.683174 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerStarted","Data":"1656b3ddd0d53c73a19e38eea07136d06df71f951c2daaacb10270e5a99dbfb5"} Jul 06 21:23:08.683265 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:08.683247 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerStarted","Data":"1f5dd7779a5842e9b6659c22b37e1ba07ddab2cb17c5687a48b59f56f619aef0"} Jul 06 21:23:08.816052 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:08.816016 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:08.886100 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:08.886063 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:10.241196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:10.241158 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:10.331187 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:10.331154 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:10.692792 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:10.692765 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerStarted","Data":"925cf7c3b65e26896750be155eea2d446a099fbf84bc5ccf139eed29e095ff3b"} Jul 06 21:23:10.692912 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:10.692842 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerStarted","Data":"ccda1f0f1d3222ef1fb2a7a69f99284ed4825417b0c8999e28122efd30fa57a1"} Jul 06 21:23:10.692912 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:10.692885 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerStarted","Data":"06b53d586fc50661f8683888ee94b3af2fffbc2d2322fc8e1a171f69e078de9e"} Jul 06 21:23:11.077146 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:11.077111 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:11.702151 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:11.702118 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerStarted","Data":"61780ac79e29c639a9dc87e25ad3b2569715fb256d73649c3b4b6317e096cd64"} Jul 06 21:23:11.738967 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:11.738902 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.082743634 podStartE2EDuration="9.73888655s" totalImagesPullingTime="6.634779388s" totalInitContainerRuntime="21.363528ms" isStatefulPod=false podCreationTimestamp="2026-07-06 21:23:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:23:11.736449192 +0000 UTC m=+99.488072257" watchObservedRunningTime="2026-07-06 21:23:11.73888655 +0000 UTC m=+99.490509614" Jul 06 21:23:13.281575 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:13.281550 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:23:14.671092 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:14.671058 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:14.772620 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:14.772568 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:17.880614 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:17.880582 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:18.004047 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:18.004007 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:18.585353 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:18.585326 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zx6nz" Jul 06 21:23:20.498027 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:20.497994 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:20.599193 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:20.599155 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:29.083868 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:29.083832 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:29.174044 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:29.174013 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:32.553121 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:32.553090 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:35.499118 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:35.499085 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:35.572874 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:35.572846 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:48.909434 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:48.909395 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:49.022193 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:49.022156 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:54.661461 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:54.661424 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:23:54.751585 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:23:54.751552 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:03.282329 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:03.282292 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:03.301114 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:03.301092 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:03.900115 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:03.900087 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:12.869211 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:12.869176 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:12.952918 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:12.952882 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:21.487198 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.487162 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 21:24:21.487592 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.487476 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="prometheus" containerID="cri-o://1f5dd7779a5842e9b6659c22b37e1ba07ddab2cb17c5687a48b59f56f619aef0" gracePeriod=600 Jul 06 21:24:21.487592 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.487544 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="kube-rbac-proxy" containerID="cri-o://925cf7c3b65e26896750be155eea2d446a099fbf84bc5ccf139eed29e095ff3b" gracePeriod=600 Jul 06 21:24:21.487728 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.487565 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="config-reloader" containerID="cri-o://1656b3ddd0d53c73a19e38eea07136d06df71f951c2daaacb10270e5a99dbfb5" gracePeriod=600 Jul 06 21:24:21.487728 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.487587 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="kube-rbac-proxy-web" containerID="cri-o://ccda1f0f1d3222ef1fb2a7a69f99284ed4825417b0c8999e28122efd30fa57a1" gracePeriod=600 Jul 06 21:24:21.487728 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.487620 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="kube-rbac-proxy-thanos" containerID="cri-o://61780ac79e29c639a9dc87e25ad3b2569715fb256d73649c3b4b6317e096cd64" gracePeriod=600 Jul 06 21:24:21.487728 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.487624 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="thanos-sidecar" containerID="cri-o://06b53d586fc50661f8683888ee94b3af2fffbc2d2322fc8e1a171f69e078de9e" gracePeriod=600 Jul 06 21:24:21.952982 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.952933 2573 generic.go:356] "Generic (PLEG): container finished" podID="4931891e-2c58-4301-85b8-86a9850e5500" containerID="61780ac79e29c639a9dc87e25ad3b2569715fb256d73649c3b4b6317e096cd64" exitCode=0 Jul 06 21:24:21.952982 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.952974 2573 generic.go:356] "Generic (PLEG): container finished" podID="4931891e-2c58-4301-85b8-86a9850e5500" containerID="925cf7c3b65e26896750be155eea2d446a099fbf84bc5ccf139eed29e095ff3b" exitCode=0 Jul 06 21:24:21.952982 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.952981 2573 generic.go:356] "Generic (PLEG): container finished" podID="4931891e-2c58-4301-85b8-86a9850e5500" containerID="06b53d586fc50661f8683888ee94b3af2fffbc2d2322fc8e1a171f69e078de9e" exitCode=0 Jul 06 21:24:21.952982 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.952990 2573 generic.go:356] "Generic (PLEG): container finished" podID="4931891e-2c58-4301-85b8-86a9850e5500" containerID="1656b3ddd0d53c73a19e38eea07136d06df71f951c2daaacb10270e5a99dbfb5" exitCode=0 Jul 06 21:24:21.953247 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.953000 2573 generic.go:356] "Generic (PLEG): container finished" podID="4931891e-2c58-4301-85b8-86a9850e5500" containerID="1f5dd7779a5842e9b6659c22b37e1ba07ddab2cb17c5687a48b59f56f619aef0" exitCode=0 Jul 06 21:24:21.953247 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.953009 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerDied","Data":"61780ac79e29c639a9dc87e25ad3b2569715fb256d73649c3b4b6317e096cd64"} Jul 06 21:24:21.953247 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.953070 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerDied","Data":"925cf7c3b65e26896750be155eea2d446a099fbf84bc5ccf139eed29e095ff3b"} Jul 06 21:24:21.953247 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.953097 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerDied","Data":"06b53d586fc50661f8683888ee94b3af2fffbc2d2322fc8e1a171f69e078de9e"} Jul 06 21:24:21.953247 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.953120 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerDied","Data":"1656b3ddd0d53c73a19e38eea07136d06df71f951c2daaacb10270e5a99dbfb5"} Jul 06 21:24:21.953247 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:21.953144 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerDied","Data":"1f5dd7779a5842e9b6659c22b37e1ba07ddab2cb17c5687a48b59f56f619aef0"} Jul 06 21:24:22.733134 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.733111 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:22.829961 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.829871 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-2\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.829961 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.829927 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-1\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830180 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.829973 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8zq7\" (UniqueName: \"kubernetes.io/projected/4931891e-2c58-4301-85b8-86a9850e5500-kube-api-access-b8zq7\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830180 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830008 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-serving-certs-ca-bundle\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830180 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830032 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830180 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830060 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-0\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830180 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830088 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-grpc-tls\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830180 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830113 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-kubelet-serving-ca-bundle\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830180 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830159 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-tls\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830511 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830184 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-metrics-client-ca\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830511 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830213 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830511 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830242 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-thanos-prometheus-http-client-file\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830511 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830266 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4931891e-2c58-4301-85b8-86a9850e5500-tls-assets\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830511 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830302 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-kube-rbac-proxy\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830511 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830328 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4931891e-2c58-4301-85b8-86a9850e5500-config-out\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830511 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830329 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:24:22.830511 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830361 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-metrics-client-certs\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830511 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830390 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-web-config\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830511 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830423 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-config\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830511 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830469 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-trusted-ca-bundle\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830511 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830501 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-db\") pod \"4931891e-2c58-4301-85b8-86a9850e5500\" (UID: \"4931891e-2c58-4301-85b8-86a9850e5500\") " Jul 06 21:24:22.830511 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830502 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:24:22.831523 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830560 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:24:22.831523 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830718 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.831523 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830725 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:24:22.831523 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830737 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.831523 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.830755 2573 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.831523 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.831148 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:24:22.832142 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.831556 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:24:22.832142 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.832032 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:24:22.832398 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.832372 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 21:24:22.833260 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.833229 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:24:22.833361 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.833344 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:24:22.833666 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.833639 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4931891e-2c58-4301-85b8-86a9850e5500-config-out" (OuterVolumeSpecName: "config-out") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 21:24:22.834208 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.834167 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4931891e-2c58-4301-85b8-86a9850e5500-kube-api-access-b8zq7" (OuterVolumeSpecName: "kube-api-access-b8zq7") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "kube-api-access-b8zq7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:24:22.834764 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.834719 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:24:22.834861 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.834770 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4931891e-2c58-4301-85b8-86a9850e5500-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:24:22.834861 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.834779 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:24:22.835183 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.835156 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:24:22.835359 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.835335 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:24:22.835441 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.835423 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-config" (OuterVolumeSpecName: "config") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:24:22.836673 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.836647 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:24:22.844016 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.843989 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-web-config" (OuterVolumeSpecName: "web-config") pod "4931891e-2c58-4301-85b8-86a9850e5500" (UID: "4931891e-2c58-4301-85b8-86a9850e5500"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:24:22.931572 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931526 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-db\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931572 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931569 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8zq7\" (UniqueName: \"kubernetes.io/projected/4931891e-2c58-4301-85b8-86a9850e5500-kube-api-access-b8zq7\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931572 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931581 2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931572 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931590 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931814 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931599 2573 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-grpc-tls\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931814 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931607 2573 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931814 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931616 2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931814 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931624 2573 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-configmap-metrics-client-ca\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931814 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931634 2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931814 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931642 2573 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931814 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931650 2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4931891e-2c58-4301-85b8-86a9850e5500-tls-assets\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931814 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931658 2573 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-kube-rbac-proxy\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931814 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931666 2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4931891e-2c58-4301-85b8-86a9850e5500-config-out\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931814 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931674 2573 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-secret-metrics-client-certs\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931814 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931682 2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-web-config\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931814 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931689 2573 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4931891e-2c58-4301-85b8-86a9850e5500-config\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.931814 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.931699 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4931891e-2c58-4301-85b8-86a9850e5500-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:24:22.959932 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.959907 2573 generic.go:356] "Generic (PLEG): container finished" podID="4931891e-2c58-4301-85b8-86a9850e5500" containerID="ccda1f0f1d3222ef1fb2a7a69f99284ed4825417b0c8999e28122efd30fa57a1" exitCode=0 Jul 06 21:24:22.960063 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.959988 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerDied","Data":"ccda1f0f1d3222ef1fb2a7a69f99284ed4825417b0c8999e28122efd30fa57a1"} Jul 06 21:24:22.960063 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.960056 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4931891e-2c58-4301-85b8-86a9850e5500","Type":"ContainerDied","Data":"867af4d80cde9d0fcf2920ed43da436422558434f0811b3b3ed2a5065409fe39"} Jul 06 21:24:22.960138 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.959999 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:22.960138 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.960092 2573 scope.go:117] "RemoveContainer" containerID="61780ac79e29c639a9dc87e25ad3b2569715fb256d73649c3b4b6317e096cd64" Jul 06 21:24:22.968252 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.968231 2573 scope.go:117] "RemoveContainer" containerID="925cf7c3b65e26896750be155eea2d446a099fbf84bc5ccf139eed29e095ff3b" Jul 06 21:24:22.974493 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.974478 2573 scope.go:117] "RemoveContainer" containerID="ccda1f0f1d3222ef1fb2a7a69f99284ed4825417b0c8999e28122efd30fa57a1" Jul 06 21:24:22.980545 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.980531 2573 scope.go:117] "RemoveContainer" containerID="06b53d586fc50661f8683888ee94b3af2fffbc2d2322fc8e1a171f69e078de9e" Jul 06 21:24:22.985548 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.985530 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 21:24:22.987065 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.987049 2573 scope.go:117] "RemoveContainer" containerID="1656b3ddd0d53c73a19e38eea07136d06df71f951c2daaacb10270e5a99dbfb5" Jul 06 21:24:22.994591 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.994574 2573 scope.go:117] "RemoveContainer" containerID="1f5dd7779a5842e9b6659c22b37e1ba07ddab2cb17c5687a48b59f56f619aef0" Jul 06 21:24:22.996703 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:22.996686 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 21:24:23.001224 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.001209 2573 scope.go:117] "RemoveContainer" containerID="58a0a3061940bb5bd7c3044b493755ae7fc95ae2f4b3835138f8d5b246493ae4" Jul 06 21:24:23.007112 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.007096 2573 scope.go:117] "RemoveContainer" containerID="61780ac79e29c639a9dc87e25ad3b2569715fb256d73649c3b4b6317e096cd64" Jul 06 21:24:23.007333 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:24:23.007314 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61780ac79e29c639a9dc87e25ad3b2569715fb256d73649c3b4b6317e096cd64\": container with ID starting with 61780ac79e29c639a9dc87e25ad3b2569715fb256d73649c3b4b6317e096cd64 not found: ID does not exist" containerID="61780ac79e29c639a9dc87e25ad3b2569715fb256d73649c3b4b6317e096cd64" Jul 06 21:24:23.007381 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.007341 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61780ac79e29c639a9dc87e25ad3b2569715fb256d73649c3b4b6317e096cd64"} err="failed to get container status \"61780ac79e29c639a9dc87e25ad3b2569715fb256d73649c3b4b6317e096cd64\": rpc error: code = NotFound desc = could not find container \"61780ac79e29c639a9dc87e25ad3b2569715fb256d73649c3b4b6317e096cd64\": container with ID starting with 61780ac79e29c639a9dc87e25ad3b2569715fb256d73649c3b4b6317e096cd64 not found: ID does not exist" Jul 06 21:24:23.007381 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.007371 2573 scope.go:117] "RemoveContainer" containerID="925cf7c3b65e26896750be155eea2d446a099fbf84bc5ccf139eed29e095ff3b" Jul 06 21:24:23.007580 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:24:23.007565 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"925cf7c3b65e26896750be155eea2d446a099fbf84bc5ccf139eed29e095ff3b\": container with ID starting with 925cf7c3b65e26896750be155eea2d446a099fbf84bc5ccf139eed29e095ff3b not found: ID does not exist" containerID="925cf7c3b65e26896750be155eea2d446a099fbf84bc5ccf139eed29e095ff3b" Jul 06 21:24:23.007621 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.007586 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"925cf7c3b65e26896750be155eea2d446a099fbf84bc5ccf139eed29e095ff3b"} err="failed to get container status \"925cf7c3b65e26896750be155eea2d446a099fbf84bc5ccf139eed29e095ff3b\": rpc error: code = NotFound desc = could not find container \"925cf7c3b65e26896750be155eea2d446a099fbf84bc5ccf139eed29e095ff3b\": container with ID starting with 925cf7c3b65e26896750be155eea2d446a099fbf84bc5ccf139eed29e095ff3b not found: ID does not exist" Jul 06 21:24:23.007621 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.007607 2573 scope.go:117] "RemoveContainer" containerID="ccda1f0f1d3222ef1fb2a7a69f99284ed4825417b0c8999e28122efd30fa57a1" Jul 06 21:24:23.007782 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:24:23.007767 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccda1f0f1d3222ef1fb2a7a69f99284ed4825417b0c8999e28122efd30fa57a1\": container with ID starting with ccda1f0f1d3222ef1fb2a7a69f99284ed4825417b0c8999e28122efd30fa57a1 not found: ID does not exist" containerID="ccda1f0f1d3222ef1fb2a7a69f99284ed4825417b0c8999e28122efd30fa57a1" Jul 06 21:24:23.007826 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.007787 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccda1f0f1d3222ef1fb2a7a69f99284ed4825417b0c8999e28122efd30fa57a1"} err="failed to get container status \"ccda1f0f1d3222ef1fb2a7a69f99284ed4825417b0c8999e28122efd30fa57a1\": rpc error: code = NotFound desc = could not find container \"ccda1f0f1d3222ef1fb2a7a69f99284ed4825417b0c8999e28122efd30fa57a1\": container with ID starting with ccda1f0f1d3222ef1fb2a7a69f99284ed4825417b0c8999e28122efd30fa57a1 not found: ID does not exist" Jul 06 21:24:23.007826 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.007800 2573 scope.go:117] "RemoveContainer" containerID="06b53d586fc50661f8683888ee94b3af2fffbc2d2322fc8e1a171f69e078de9e" Jul 06 21:24:23.008049 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:24:23.008034 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06b53d586fc50661f8683888ee94b3af2fffbc2d2322fc8e1a171f69e078de9e\": container with ID starting with 06b53d586fc50661f8683888ee94b3af2fffbc2d2322fc8e1a171f69e078de9e not found: ID does not exist" containerID="06b53d586fc50661f8683888ee94b3af2fffbc2d2322fc8e1a171f69e078de9e" Jul 06 21:24:23.008111 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.008052 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b53d586fc50661f8683888ee94b3af2fffbc2d2322fc8e1a171f69e078de9e"} err="failed to get container status \"06b53d586fc50661f8683888ee94b3af2fffbc2d2322fc8e1a171f69e078de9e\": rpc error: code = NotFound desc = could not find container \"06b53d586fc50661f8683888ee94b3af2fffbc2d2322fc8e1a171f69e078de9e\": container with ID starting with 06b53d586fc50661f8683888ee94b3af2fffbc2d2322fc8e1a171f69e078de9e not found: ID does not exist" Jul 06 21:24:23.008111 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.008065 2573 scope.go:117] "RemoveContainer" containerID="1656b3ddd0d53c73a19e38eea07136d06df71f951c2daaacb10270e5a99dbfb5" Jul 06 21:24:23.008308 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:24:23.008291 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1656b3ddd0d53c73a19e38eea07136d06df71f951c2daaacb10270e5a99dbfb5\": container with ID starting with 1656b3ddd0d53c73a19e38eea07136d06df71f951c2daaacb10270e5a99dbfb5 not found: ID does not exist" containerID="1656b3ddd0d53c73a19e38eea07136d06df71f951c2daaacb10270e5a99dbfb5" Jul 06 21:24:23.008349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.008311 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1656b3ddd0d53c73a19e38eea07136d06df71f951c2daaacb10270e5a99dbfb5"} err="failed to get container status \"1656b3ddd0d53c73a19e38eea07136d06df71f951c2daaacb10270e5a99dbfb5\": rpc error: code = NotFound desc = could not find container \"1656b3ddd0d53c73a19e38eea07136d06df71f951c2daaacb10270e5a99dbfb5\": container with ID starting with 1656b3ddd0d53c73a19e38eea07136d06df71f951c2daaacb10270e5a99dbfb5 not found: ID does not exist" Jul 06 21:24:23.008349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.008326 2573 scope.go:117] "RemoveContainer" containerID="1f5dd7779a5842e9b6659c22b37e1ba07ddab2cb17c5687a48b59f56f619aef0" Jul 06 21:24:23.008535 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:24:23.008522 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f5dd7779a5842e9b6659c22b37e1ba07ddab2cb17c5687a48b59f56f619aef0\": container with ID starting with 1f5dd7779a5842e9b6659c22b37e1ba07ddab2cb17c5687a48b59f56f619aef0 not found: ID does not exist" containerID="1f5dd7779a5842e9b6659c22b37e1ba07ddab2cb17c5687a48b59f56f619aef0" Jul 06 21:24:23.008575 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.008538 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f5dd7779a5842e9b6659c22b37e1ba07ddab2cb17c5687a48b59f56f619aef0"} err="failed to get container status \"1f5dd7779a5842e9b6659c22b37e1ba07ddab2cb17c5687a48b59f56f619aef0\": rpc error: code = NotFound desc = could not find container \"1f5dd7779a5842e9b6659c22b37e1ba07ddab2cb17c5687a48b59f56f619aef0\": container with ID starting with 1f5dd7779a5842e9b6659c22b37e1ba07ddab2cb17c5687a48b59f56f619aef0 not found: ID does not exist" Jul 06 21:24:23.008575 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.008549 2573 scope.go:117] "RemoveContainer" containerID="58a0a3061940bb5bd7c3044b493755ae7fc95ae2f4b3835138f8d5b246493ae4" Jul 06 21:24:23.008744 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:24:23.008727 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58a0a3061940bb5bd7c3044b493755ae7fc95ae2f4b3835138f8d5b246493ae4\": container with ID starting with 58a0a3061940bb5bd7c3044b493755ae7fc95ae2f4b3835138f8d5b246493ae4 not found: ID does not exist" containerID="58a0a3061940bb5bd7c3044b493755ae7fc95ae2f4b3835138f8d5b246493ae4" Jul 06 21:24:23.008784 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.008749 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58a0a3061940bb5bd7c3044b493755ae7fc95ae2f4b3835138f8d5b246493ae4"} err="failed to get container status \"58a0a3061940bb5bd7c3044b493755ae7fc95ae2f4b3835138f8d5b246493ae4\": rpc error: code = NotFound desc = could not find container \"58a0a3061940bb5bd7c3044b493755ae7fc95ae2f4b3835138f8d5b246493ae4\": container with ID starting with 58a0a3061940bb5bd7c3044b493755ae7fc95ae2f4b3835138f8d5b246493ae4 not found: ID does not exist" Jul 06 21:24:23.038798 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.038779 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 21:24:23.039122 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039104 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="init-config-reloader" Jul 06 21:24:23.039225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039125 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="init-config-reloader" Jul 06 21:24:23.039225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039137 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="kube-rbac-proxy-web" Jul 06 21:24:23.039225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039147 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="kube-rbac-proxy-web" Jul 06 21:24:23.039225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039155 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="kube-rbac-proxy" Jul 06 21:24:23.039225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039164 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="kube-rbac-proxy" Jul 06 21:24:23.039225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039191 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="prometheus" Jul 06 21:24:23.039225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039199 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="prometheus" Jul 06 21:24:23.039225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039208 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="config-reloader" Jul 06 21:24:23.039225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039217 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="config-reloader" Jul 06 21:24:23.039225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039225 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="thanos-sidecar" Jul 06 21:24:23.039710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039233 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="thanos-sidecar" Jul 06 21:24:23.039710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039245 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="kube-rbac-proxy-thanos" Jul 06 21:24:23.039710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039253 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="kube-rbac-proxy-thanos" Jul 06 21:24:23.039710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039321 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="thanos-sidecar" Jul 06 21:24:23.039710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039334 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="kube-rbac-proxy-thanos" Jul 06 21:24:23.039710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039344 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="prometheus" Jul 06 21:24:23.039710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039354 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="config-reloader" Jul 06 21:24:23.039710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039362 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="kube-rbac-proxy" Jul 06 21:24:23.039710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.039370 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4931891e-2c58-4301-85b8-86a9850e5500" containerName="kube-rbac-proxy-web" Jul 06 21:24:23.044496 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.044480 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.049976 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.049935 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 21:24:23.050097 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.050000 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 21:24:23.050216 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.050115 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 21:24:23.050296 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.050217 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 21:24:23.050296 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.050266 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 21:24:23.050529 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.050353 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 21:24:23.050808 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.050637 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 21:24:23.050899 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.050885 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 21:24:23.051168 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.051128 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pxbln\"" Jul 06 21:24:23.051336 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.051284 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 21:24:23.051428 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.051341 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 21:24:23.051428 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.051349 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 21:24:23.051541 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.051341 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4q8v00j0h5bji\"" Jul 06 21:24:23.051541 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.051436 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 21:24:23.053053 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.052811 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 21:24:23.056975 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.056294 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 21:24:23.058924 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.058907 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 21:24:23.084477 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.084426 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 21:24:23.132882 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.132856 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.132999 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.132886 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.132999 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.132908 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.132999 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.132926 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.132999 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.132982 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-web-config\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133134 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133017 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8940ca46-00ef-45b6-a98f-8b896d4c12ba-config-out\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133134 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133042 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133134 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133061 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133134 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133079 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133134 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133123 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-config\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133285 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133148 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133285 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133163 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133285 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133182 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133285 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133210 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133285 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133226 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133285 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133253 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8940ca46-00ef-45b6-a98f-8b896d4c12ba-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133451 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133298 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133451 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133317 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133451 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133352 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mplln\" (UniqueName: \"kubernetes.io/projected/8940ca46-00ef-45b6-a98f-8b896d4c12ba-kube-api-access-mplln\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.133451 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.133368 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.233804 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.233774 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8940ca46-00ef-45b6-a98f-8b896d4c12ba-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.233804 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.233806 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234010 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.233823 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234010 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.233845 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mplln\" (UniqueName: \"kubernetes.io/projected/8940ca46-00ef-45b6-a98f-8b896d4c12ba-kube-api-access-mplln\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234010 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.233870 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234010 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.233894 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234028 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234079 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234115 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234142 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-web-config\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234176 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8940ca46-00ef-45b6-a98f-8b896d4c12ba-config-out\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234437 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234200 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234437 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234229 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234437 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234257 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234437 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234294 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-config\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234437 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234317 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234437 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234343 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234437 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234374 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234437 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234413 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234828 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234437 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234828 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234754 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.234828 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234763 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.235006 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.234884 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.235586 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.235558 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.235768 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.235741 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.236837 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.236806 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8940ca46-00ef-45b6-a98f-8b896d4c12ba-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.237079 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.237056 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.237380 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.237356 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.237478 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.237409 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.237849 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.237798 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.238372 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.238055 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-config\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.238372 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.238145 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.238372 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.238219 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.238372 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.238361 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.239042 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.239020 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8940ca46-00ef-45b6-a98f-8b896d4c12ba-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.239341 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.239322 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.239791 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.239771 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-web-config\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.239921 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.239903 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8940ca46-00ef-45b6-a98f-8b896d4c12ba-config-out\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.240077 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.240058 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8940ca46-00ef-45b6-a98f-8b896d4c12ba-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.246275 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.246256 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mplln\" (UniqueName: \"kubernetes.io/projected/8940ca46-00ef-45b6-a98f-8b896d4c12ba-kube-api-access-mplln\") pod \"prometheus-k8s-0\" (UID: \"8940ca46-00ef-45b6-a98f-8b896d4c12ba\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.354049 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.353976 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:23.506574 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:24:23.506548 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8940ca46_00ef_45b6_a98f_8b896d4c12ba.slice/crio-b09ae143fe897cd65f2133efc9cacf9e3f9e2d85354ee970027912dd7e241b5e WatchSource:0}: Error finding container b09ae143fe897cd65f2133efc9cacf9e3f9e2d85354ee970027912dd7e241b5e: Status 404 returned error can't find the container with id b09ae143fe897cd65f2133efc9cacf9e3f9e2d85354ee970027912dd7e241b5e Jul 06 21:24:23.508283 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.508260 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 21:24:23.964893 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.964799 2573 generic.go:356] "Generic (PLEG): container finished" podID="8940ca46-00ef-45b6-a98f-8b896d4c12ba" containerID="95dac815e34de908bc14a2d2a8d771cfe1b61371d9a870ee9409f6adf382dcf5" exitCode=0 Jul 06 21:24:23.965317 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.964883 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8940ca46-00ef-45b6-a98f-8b896d4c12ba","Type":"ContainerDied","Data":"95dac815e34de908bc14a2d2a8d771cfe1b61371d9a870ee9409f6adf382dcf5"} Jul 06 21:24:23.965317 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:23.964961 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8940ca46-00ef-45b6-a98f-8b896d4c12ba","Type":"ContainerStarted","Data":"b09ae143fe897cd65f2133efc9cacf9e3f9e2d85354ee970027912dd7e241b5e"} Jul 06 21:24:24.218066 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:24.218044 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4931891e-2c58-4301-85b8-86a9850e5500" path="/var/lib/kubelet/pods/4931891e-2c58-4301-85b8-86a9850e5500/volumes" Jul 06 21:24:24.784381 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:24.784342 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:24.863990 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:24.863962 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:24.972681 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:24.972652 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8940ca46-00ef-45b6-a98f-8b896d4c12ba","Type":"ContainerStarted","Data":"113738513f30ef345a28c9a1c916ce02bb5d7043c2f05ddb0bf199bfc5b6fa5c"} Jul 06 21:24:24.973054 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:24.972738 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8940ca46-00ef-45b6-a98f-8b896d4c12ba","Type":"ContainerStarted","Data":"79c97aecf418071fa59c8124d64f0eaa9f1cbb5a7d483e9c23703feaee59391b"} Jul 06 21:24:24.973054 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:24.972769 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8940ca46-00ef-45b6-a98f-8b896d4c12ba","Type":"ContainerStarted","Data":"b339643143450938fcc3c15448620c161dcd0c7bc41b856858e3095e4e3cbc83"} Jul 06 21:24:24.973054 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:24.972791 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8940ca46-00ef-45b6-a98f-8b896d4c12ba","Type":"ContainerStarted","Data":"4b05bbb4943a0b00e7ce3e4655d69c95b98340f4e4874d216c3bccf688c589c0"} Jul 06 21:24:24.973054 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:24.972814 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8940ca46-00ef-45b6-a98f-8b896d4c12ba","Type":"ContainerStarted","Data":"cf8d662e7189d229be89da1edfb53baa707a24d64a07eb72d276d217f2260c41"} Jul 06 21:24:24.973054 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:24.972837 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8940ca46-00ef-45b6-a98f-8b896d4c12ba","Type":"ContainerStarted","Data":"d382e9ebba947b4d708054392dcf8de5b12fd0717ae34e9dc0e290a75549f30d"} Jul 06 21:24:25.015230 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:25.015152 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.993005552 podStartE2EDuration="2.015138117s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.132565ms" isStatefulPod=false podCreationTimestamp="2026-07-06 21:24:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:24:25.011684179 +0000 UTC m=+172.763307245" watchObservedRunningTime="2026-07-06 21:24:25.015138117 +0000 UTC m=+172.766761181" Jul 06 21:24:26.079081 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:26.079049 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:26.185294 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:26.185262 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:28.354917 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:28.354881 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:24:35.585326 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:35.585294 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:35.683687 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:35.683655 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:37.253710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:37.253675 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:24:37.392067 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:24:37.392037 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:01.012785 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:01.012752 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:01.109549 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:01.109516 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:23.354460 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:23.354417 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:25:23.369350 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:23.369324 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:25:24.192474 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:24.192444 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 21:25:24.757385 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:24.757340 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:24.845363 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:24.845328 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:28.646828 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:28.646792 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:28.841703 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:28.841669 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:32.620995 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:32.620958 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:32.747100 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:32.747063 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:34.778335 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:34.778293 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:34.930571 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:34.930533 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:37.094145 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:37.094112 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:37.526978 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:37.526878 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:38.997994 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:38.997929 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:39.100919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:39.100884 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:45.304316 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:45.304256 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:45.754136 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:45.754041 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:45.852264 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:45.852230 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:50.962437 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:50.962393 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:51.072673 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:51.072640 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:25:53.628191 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:25:53.628142 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:00.679192 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:00.679113 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:00.737838 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:00.737809 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:04.571163 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:04.571126 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:04.670710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:04.670682 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:08.865444 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:08.865412 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:08.975093 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:08.975059 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:09.489455 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:09.489422 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:09.742023 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:09.741932 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:09.854244 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:09.854216 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:10.008311 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.008237 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq"] Jul 06 21:26:10.011395 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.011376 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:10.015254 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:26:10.015226 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-131-78.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-78.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 06 21:26:10.015357 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:26:10.015333 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-131-78.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-78.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 06 21:26:10.015405 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:26:10.015359 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-dockercfg-qmkl2\" is forbidden: User \"system:node:ip-10-0-131-78.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-78.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qmkl2\"" type="*v1.Secret" Jul 06 21:26:10.015515 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:26:10.015493 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-service-cert\" is forbidden: User \"system:node:ip-10-0-131-78.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-78.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret" Jul 06 21:26:10.015756 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:26:10.015734 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-webhook-cert\" is forbidden: User \"system:node:ip-10-0-131-78.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-78.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" type="*v1.Secret" Jul 06 21:26:10.015810 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:26:10.015780 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" is forbidden: User \"system:node:ip-10-0-131-78.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-78.ec2.internal' and this object" podUID="a48ed58f-fbd7-48e4-82fd-fd935011c29e" pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:10.060185 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.060161 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq"] Jul 06 21:26:10.115977 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.115935 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a48ed58f-fbd7-48e4-82fd-fd935011c29e-tmp\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:10.116107 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.115983 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a48ed58f-fbd7-48e4-82fd-fd935011c29e-webhook-cert\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:10.116107 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.116009 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a48ed58f-fbd7-48e4-82fd-fd935011c29e-manifests\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:10.116225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.116135 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a48ed58f-fbd7-48e4-82fd-fd935011c29e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:10.116225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.116179 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgmqt\" (UniqueName: \"kubernetes.io/projected/a48ed58f-fbd7-48e4-82fd-fd935011c29e-kube-api-access-mgmqt\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:10.204826 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.204794 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:10.217450 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.217423 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a48ed58f-fbd7-48e4-82fd-fd935011c29e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:10.217565 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.217456 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgmqt\" (UniqueName: \"kubernetes.io/projected/a48ed58f-fbd7-48e4-82fd-fd935011c29e-kube-api-access-mgmqt\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:10.217565 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.217489 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a48ed58f-fbd7-48e4-82fd-fd935011c29e-tmp\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:10.217675 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.217598 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a48ed58f-fbd7-48e4-82fd-fd935011c29e-webhook-cert\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:10.217675 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.217635 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a48ed58f-fbd7-48e4-82fd-fd935011c29e-manifests\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:10.217987 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.217970 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a48ed58f-fbd7-48e4-82fd-fd935011c29e-manifests\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:10.219575 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.219556 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a48ed58f-fbd7-48e4-82fd-fd935011c29e-tmp\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:10.302979 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.302938 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:10.409721 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.409686 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:10.511502 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.511470 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:10.678672 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.678591 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:10.781066 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:10.781039 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:11.005313 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:11.005244 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 21:26:11.064235 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:11.064206 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 21:26:11.070071 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:11.070051 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a48ed58f-fbd7-48e4-82fd-fd935011c29e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:11.070165 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:11.070092 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a48ed58f-fbd7-48e4-82fd-fd935011c29e-webhook-cert\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:11.161060 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:11.161036 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 21:26:11.218456 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:11.218431 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 21:26:11.223277 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:11.223253 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgmqt\" (UniqueName: \"kubernetes.io/projected/a48ed58f-fbd7-48e4-82fd-fd935011c29e-kube-api-access-mgmqt\") pod \"opendatahub-operator-controller-manager-757974bfb6-hq8xq\" (UID: \"a48ed58f-fbd7-48e4-82fd-fd935011c29e\") " pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:11.237237 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:11.237212 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qmkl2\"" Jul 06 21:26:11.520682 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:11.520650 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:11.651622 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:11.651586 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq"] Jul 06 21:26:11.654613 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:26:11.654580 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda48ed58f_fbd7_48e4_82fd_fd935011c29e.slice/crio-1dff93ce0e824420ed14b25ae75d29b57292fd5c1ba2fe6482180bfabb9856d6 WatchSource:0}: Error finding container 1dff93ce0e824420ed14b25ae75d29b57292fd5c1ba2fe6482180bfabb9856d6: Status 404 returned error can't find the container with id 1dff93ce0e824420ed14b25ae75d29b57292fd5c1ba2fe6482180bfabb9856d6 Jul 06 21:26:12.339569 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:12.339532 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" event={"ID":"a48ed58f-fbd7-48e4-82fd-fd935011c29e","Type":"ContainerStarted","Data":"1dff93ce0e824420ed14b25ae75d29b57292fd5c1ba2fe6482180bfabb9856d6"} Jul 06 21:26:15.350747 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:15.350718 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" event={"ID":"a48ed58f-fbd7-48e4-82fd-fd935011c29e","Type":"ContainerStarted","Data":"8161e568e624d01e406fb98fab2a305a3c8c34e772629489f742fd578086849a"} Jul 06 21:26:16.355776 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:16.355742 2573 generic.go:356] "Generic (PLEG): container finished" podID="a48ed58f-fbd7-48e4-82fd-fd935011c29e" containerID="8161e568e624d01e406fb98fab2a305a3c8c34e772629489f742fd578086849a" exitCode=0 Jul 06 21:26:16.356169 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:16.355806 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" event={"ID":"a48ed58f-fbd7-48e4-82fd-fd935011c29e","Type":"ContainerDied","Data":"8161e568e624d01e406fb98fab2a305a3c8c34e772629489f742fd578086849a"} Jul 06 21:26:17.361274 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:17.361234 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" event={"ID":"a48ed58f-fbd7-48e4-82fd-fd935011c29e","Type":"ContainerStarted","Data":"d132c257c003410cc0aae05e94b22f357c4bc62214425ff6befd9716e36c45d5"} Jul 06 21:26:17.361759 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:17.361360 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:17.386078 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:17.386027 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" podStartSLOduration=4.118961429 podStartE2EDuration="8.386012314s" totalImagesPullingTime="3.18005747s" totalInitContainerRuntime="1.086993415s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:26:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:26:17.384803141 +0000 UTC m=+285.136426205" watchObservedRunningTime="2026-07-06 21:26:17.386012314 +0000 UTC m=+285.137635378" Jul 06 21:26:19.913538 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:19.913483 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:20.011351 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:20.011293 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:21.879195 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:21.879156 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:21.978577 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:21.978538 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:28.367101 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:28.367067 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-757974bfb6-hq8xq" Jul 06 21:26:30.200686 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:30.200654 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:31.370319 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:31.370282 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:31.828371 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:31.828327 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:31.945397 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:31.945364 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:32.330406 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:32.330373 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:32.395774 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:32.395739 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:33.465712 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:33.465680 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:34.064191 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:34.064164 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:26:34.064623 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:34.064601 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:26:34.067735 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:34.067719 2573 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 21:26:37.770358 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:37.770318 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:37.871087 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:37.871059 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:38.438906 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:38.438859 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:38.678593 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:38.678561 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:38.867279 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:38.867240 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:38.936022 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:38.935995 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb"] Jul 06 21:26:38.939025 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:38.939010 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:38.943535 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:38.943509 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-qchm7\"" Jul 06 21:26:38.943643 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:38.943542 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 21:26:38.943643 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:38.943565 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 21:26:38.943643 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:38.943514 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 21:26:38.943643 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:38.943631 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 21:26:38.943839 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:38.943826 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 21:26:38.950056 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:38.950033 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb"] Jul 06 21:26:39.034388 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.034359 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e904cae-cf94-440e-8181-1305357fe59f-cert\") pod \"lws-controller-manager-d694bd4f8-gn5pb\" (UID: \"3e904cae-cf94-440e-8181-1305357fe59f\") " pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:39.034556 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.034396 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3e904cae-cf94-440e-8181-1305357fe59f-manager-config\") pod \"lws-controller-manager-d694bd4f8-gn5pb\" (UID: \"3e904cae-cf94-440e-8181-1305357fe59f\") " pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:39.034556 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.034433 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frhnk\" (UniqueName: \"kubernetes.io/projected/3e904cae-cf94-440e-8181-1305357fe59f-kube-api-access-frhnk\") pod \"lws-controller-manager-d694bd4f8-gn5pb\" (UID: \"3e904cae-cf94-440e-8181-1305357fe59f\") " pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:39.034556 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.034486 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3e904cae-cf94-440e-8181-1305357fe59f-metrics-cert\") pod \"lws-controller-manager-d694bd4f8-gn5pb\" (UID: \"3e904cae-cf94-440e-8181-1305357fe59f\") " pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:39.110195 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.110166 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np"] Jul 06 21:26:39.113216 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.113191 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.116063 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.116043 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 21:26:39.116063 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.116058 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 21:26:39.116281 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.116091 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 21:26:39.116344 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.116322 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 21:26:39.116449 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.116323 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 21:26:39.116575 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.116560 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 21:26:39.116661 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.116645 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4b4dp\"" Jul 06 21:26:39.127242 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.127219 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np"] Jul 06 21:26:39.135082 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.135060 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frhnk\" (UniqueName: \"kubernetes.io/projected/3e904cae-cf94-440e-8181-1305357fe59f-kube-api-access-frhnk\") pod \"lws-controller-manager-d694bd4f8-gn5pb\" (UID: \"3e904cae-cf94-440e-8181-1305357fe59f\") " pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:39.135183 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.135098 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3e904cae-cf94-440e-8181-1305357fe59f-metrics-cert\") pod \"lws-controller-manager-d694bd4f8-gn5pb\" (UID: \"3e904cae-cf94-440e-8181-1305357fe59f\") " pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:39.135183 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.135137 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e904cae-cf94-440e-8181-1305357fe59f-cert\") pod \"lws-controller-manager-d694bd4f8-gn5pb\" (UID: \"3e904cae-cf94-440e-8181-1305357fe59f\") " pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:39.135183 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.135157 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3e904cae-cf94-440e-8181-1305357fe59f-manager-config\") pod \"lws-controller-manager-d694bd4f8-gn5pb\" (UID: \"3e904cae-cf94-440e-8181-1305357fe59f\") " pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:39.135869 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.135845 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3e904cae-cf94-440e-8181-1305357fe59f-manager-config\") pod \"lws-controller-manager-d694bd4f8-gn5pb\" (UID: \"3e904cae-cf94-440e-8181-1305357fe59f\") " pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:39.137753 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.137731 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3e904cae-cf94-440e-8181-1305357fe59f-metrics-cert\") pod \"lws-controller-manager-d694bd4f8-gn5pb\" (UID: \"3e904cae-cf94-440e-8181-1305357fe59f\") " pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:39.137888 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.137870 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e904cae-cf94-440e-8181-1305357fe59f-cert\") pod \"lws-controller-manager-d694bd4f8-gn5pb\" (UID: \"3e904cae-cf94-440e-8181-1305357fe59f\") " pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:39.145405 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.145385 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frhnk\" (UniqueName: \"kubernetes.io/projected/3e904cae-cf94-440e-8181-1305357fe59f-kube-api-access-frhnk\") pod \"lws-controller-manager-d694bd4f8-gn5pb\" (UID: \"3e904cae-cf94-440e-8181-1305357fe59f\") " pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:39.236253 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.236225 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bvxt\" (UniqueName: \"kubernetes.io/projected/ed271837-a04a-46e4-921f-8acb8f0395f9-kube-api-access-6bvxt\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.236404 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.236273 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ed271837-a04a-46e4-921f-8acb8f0395f9-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.236404 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.236360 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ed271837-a04a-46e4-921f-8acb8f0395f9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.236493 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.236412 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed271837-a04a-46e4-921f-8acb8f0395f9-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.236493 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.236442 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed271837-a04a-46e4-921f-8acb8f0395f9-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.236493 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.236486 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ed271837-a04a-46e4-921f-8acb8f0395f9-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.236635 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.236562 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ed271837-a04a-46e4-921f-8acb8f0395f9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.248200 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.248175 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:39.337504 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.337471 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ed271837-a04a-46e4-921f-8acb8f0395f9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.337633 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.337534 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed271837-a04a-46e4-921f-8acb8f0395f9-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.337633 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.337562 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed271837-a04a-46e4-921f-8acb8f0395f9-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.337633 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.337601 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ed271837-a04a-46e4-921f-8acb8f0395f9-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.337788 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.337642 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ed271837-a04a-46e4-921f-8acb8f0395f9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.337788 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.337680 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bvxt\" (UniqueName: \"kubernetes.io/projected/ed271837-a04a-46e4-921f-8acb8f0395f9-kube-api-access-6bvxt\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.337788 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.337723 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ed271837-a04a-46e4-921f-8acb8f0395f9-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.338709 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.338681 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ed271837-a04a-46e4-921f-8acb8f0395f9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.340036 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.340003 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed271837-a04a-46e4-921f-8acb8f0395f9-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.340149 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.340126 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ed271837-a04a-46e4-921f-8acb8f0395f9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.340849 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.340824 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ed271837-a04a-46e4-921f-8acb8f0395f9-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.341070 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.341053 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ed271837-a04a-46e4-921f-8acb8f0395f9-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.353073 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.353051 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed271837-a04a-46e4-921f-8acb8f0395f9-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.353194 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.353172 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bvxt\" (UniqueName: \"kubernetes.io/projected/ed271837-a04a-46e4-921f-8acb8f0395f9-kube-api-access-6bvxt\") pod \"istiod-openshift-gateway-94bb8fbfd-gz5np\" (UID: \"ed271837-a04a-46e4-921f-8acb8f0395f9\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.408266 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.408214 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb"] Jul 06 21:26:39.411075 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:26:39.411054 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e904cae_cf94_440e_8181_1305357fe59f.slice/crio-c3bde07752ab06fed26264fb469bb48ef29fa4b9584144177e598bb8872bc3b9 WatchSource:0}: Error finding container c3bde07752ab06fed26264fb469bb48ef29fa4b9584144177e598bb8872bc3b9: Status 404 returned error can't find the container with id c3bde07752ab06fed26264fb469bb48ef29fa4b9584144177e598bb8872bc3b9 Jul 06 21:26:39.412889 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.412860 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 21:26:39.422623 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.422606 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:39.437766 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.437746 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" event={"ID":"3e904cae-cf94-440e-8181-1305357fe59f","Type":"ContainerStarted","Data":"c3bde07752ab06fed26264fb469bb48ef29fa4b9584144177e598bb8872bc3b9"} Jul 06 21:26:39.555067 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.555037 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np"] Jul 06 21:26:39.557847 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:26:39.557822 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded271837_a04a_46e4_921f_8acb8f0395f9.slice/crio-3a578dc44d018c638f38ad926f4328fc9d934ad76322d4bda53fd39f39a7dca3 WatchSource:0}: Error finding container 3a578dc44d018c638f38ad926f4328fc9d934ad76322d4bda53fd39f39a7dca3: Status 404 returned error can't find the container with id 3a578dc44d018c638f38ad926f4328fc9d934ad76322d4bda53fd39f39a7dca3 Jul 06 21:26:39.631401 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.631369 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:39.731052 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:39.730938 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:40.443982 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:40.443930 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" event={"ID":"ed271837-a04a-46e4-921f-8acb8f0395f9","Type":"ContainerStarted","Data":"3a578dc44d018c638f38ad926f4328fc9d934ad76322d4bda53fd39f39a7dca3"} Jul 06 21:26:41.895967 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:41.895914 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:42.027329 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:42.027278 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:42.100793 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:42.100743 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:42.239523 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:42.239436 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:42.352921 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:42.352886 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:42.500246 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:42.500171 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:26:43.065414 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:43.065361 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 21:26:43.065711 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:43.065472 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 21:26:43.458444 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:43.458340 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" event={"ID":"3e904cae-cf94-440e-8181-1305357fe59f","Type":"ContainerStarted","Data":"ff91af165bc7a235500cd0a4cc107ec755f545f7af5cb99aa792ba6470c56a8c"} Jul 06 21:26:43.458606 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:43.458449 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:26:43.460017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:43.459988 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" event={"ID":"ed271837-a04a-46e4-921f-8acb8f0395f9","Type":"ContainerStarted","Data":"52888a5424a0ffdef718e471baaea69d1ab0ba0a6b07eb96ac94b773ede0855a"} Jul 06 21:26:43.460139 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:43.460110 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:43.481158 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:43.481108 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" podStartSLOduration=1.776657495 podStartE2EDuration="5.481094169s" totalImagesPullingTime="3.704436674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:26:43.479973978 +0000 UTC m=+311.231597039" watchObservedRunningTime="2026-07-06 21:26:43.481094169 +0000 UTC m=+311.232717233" Jul 06 21:26:43.504180 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:43.504134 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" podStartSLOduration=0.998878233 podStartE2EDuration="4.504119583s" totalImagesPullingTime="3.50524135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:26:43.502503052 +0000 UTC m=+311.254126117" watchObservedRunningTime="2026-07-06 21:26:43.504119583 +0000 UTC m=+311.255742648" Jul 06 21:26:44.464835 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:44.464799 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gz5np" Jul 06 21:26:54.464998 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:26:54.464956 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d694bd4f8-gn5pb" Jul 06 21:27:06.661099 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:27:06.661069 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:27:09.475331 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:27:09.475297 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:27:23.202573 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:27:23.202541 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:27:31.752702 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:27:31.752621 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:27:31.849869 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:27:31.849830 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:27:31.934276 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:27:31.934246 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:27:32.681027 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:27:32.680990 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:27:32.899039 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:27:32.899004 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:27:43.743362 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:27:43.743331 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:27:52.767669 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:27:52.767616 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:27:56.948636 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:27:56.948591 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:27:59.435802 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:27:59.435768 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:27:59.606805 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:27:59.606770 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:00.841642 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:00.841599 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:03.954781 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:03.954749 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:04.361344 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:04.361313 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:04.697197 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:04.697121 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:04.786018 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:04.785990 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:07.453741 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:07.453709 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:08.760690 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:08.760641 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:08.861977 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:08.861928 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:12.065792 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:12.065755 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:12.162780 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:12.162745 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:12.541094 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:12.541068 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:12.642755 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:12.642716 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:13.879147 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:13.879115 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:14.030809 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:14.030776 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:15.949179 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:15.949136 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:16.046754 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:16.046711 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:24.490165 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:24.490121 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:24.654993 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:24.654960 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:27.113710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:27.113679 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:27.214537 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:27.214502 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:31.254889 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:31.254855 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:31.361193 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:31.361165 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:31.776060 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:31.776023 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:31.875312 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:31.875274 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:31.975680 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:31.975629 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:32.488189 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:32.488148 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:32.588567 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:32.588536 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:34.754569 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:34.754533 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:34.822622 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:34.822576 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:41.063170 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:41.063120 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:41.163811 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:41.163774 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:42.483163 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:42.483126 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:42.582940 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:42.582902 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:43.207013 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:43.206976 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:43.312690 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:43.312654 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:45.812200 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:45.812150 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:45.914301 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:45.914268 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:49.014391 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:49.014360 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:49.115246 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:49.115215 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:52.616314 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:52.616281 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:52.715562 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:52.715513 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:28:54.847165 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:28:54.847092 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:01.137762 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:01.137731 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:01.238864 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:01.238837 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:06.748764 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:06.748717 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:07.244881 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:07.244849 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:11.581064 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.581025 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h77bg"] Jul 06 21:29:11.584035 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.584017 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" Jul 06 21:29:11.588214 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.588191 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 21:29:11.588325 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.588220 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 21:29:11.588939 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.588921 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 21:29:11.589707 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.589687 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zkjzz\"" Jul 06 21:29:11.590798 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.590376 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:11.600858 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.600836 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h77bg"] Jul 06 21:29:11.605521 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.605500 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkth6\" (UniqueName: \"kubernetes.io/projected/fa8df390-7638-45f6-bdc4-9923cd9ae371-kube-api-access-nkth6\") pod \"limitador-limitador-64c94fd64f-h77bg\" (UID: \"fa8df390-7638-45f6-bdc4-9923cd9ae371\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" Jul 06 21:29:11.605635 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.605548 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fa8df390-7638-45f6-bdc4-9923cd9ae371-config-file\") pod \"limitador-limitador-64c94fd64f-h77bg\" (UID: \"fa8df390-7638-45f6-bdc4-9923cd9ae371\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" Jul 06 21:29:11.661347 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.661308 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h77bg"] Jul 06 21:29:11.700761 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.700729 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:11.706808 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.706790 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkth6\" (UniqueName: \"kubernetes.io/projected/fa8df390-7638-45f6-bdc4-9923cd9ae371-kube-api-access-nkth6\") pod \"limitador-limitador-64c94fd64f-h77bg\" (UID: \"fa8df390-7638-45f6-bdc4-9923cd9ae371\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" Jul 06 21:29:11.706867 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.706826 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fa8df390-7638-45f6-bdc4-9923cd9ae371-config-file\") pod \"limitador-limitador-64c94fd64f-h77bg\" (UID: \"fa8df390-7638-45f6-bdc4-9923cd9ae371\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" Jul 06 21:29:11.707455 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.707439 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fa8df390-7638-45f6-bdc4-9923cd9ae371-config-file\") pod \"limitador-limitador-64c94fd64f-h77bg\" (UID: \"fa8df390-7638-45f6-bdc4-9923cd9ae371\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" Jul 06 21:29:11.715709 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.715689 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkth6\" (UniqueName: \"kubernetes.io/projected/fa8df390-7638-45f6-bdc4-9923cd9ae371-kube-api-access-nkth6\") pod \"limitador-limitador-64c94fd64f-h77bg\" (UID: \"fa8df390-7638-45f6-bdc4-9923cd9ae371\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" Jul 06 21:29:11.804145 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.804110 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:11.894635 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:11.894557 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" Jul 06 21:29:12.031653 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:12.031631 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h77bg"] Jul 06 21:29:12.033582 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:29:12.033550 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa8df390_7638_45f6_bdc4_9923cd9ae371.slice/crio-0215128288c65336c0151986114a01a09f9b030ea58bb2283a172e39f2f1b762 WatchSource:0}: Error finding container 0215128288c65336c0151986114a01a09f9b030ea58bb2283a172e39f2f1b762: Status 404 returned error can't find the container with id 0215128288c65336c0151986114a01a09f9b030ea58bb2283a172e39f2f1b762 Jul 06 21:29:12.465982 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:12.465930 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:12.567403 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:12.567352 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:13.006358 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:13.006279 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" event={"ID":"fa8df390-7638-45f6-bdc4-9923cd9ae371","Type":"ContainerStarted","Data":"0215128288c65336c0151986114a01a09f9b030ea58bb2283a172e39f2f1b762"} Jul 06 21:29:15.016188 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:15.016147 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" event={"ID":"fa8df390-7638-45f6-bdc4-9923cd9ae371","Type":"ContainerStarted","Data":"34d3d25ef43d71757e32f6663d809daad7fc0cd8fca6d31151e38b1d67e5ec4f"} Jul 06 21:29:15.016614 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:15.016292 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" Jul 06 21:29:15.035350 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:15.035304 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" podStartSLOduration=1.21311213 podStartE2EDuration="4.035290383s" totalImagesPullingTime="2.822178253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:29:15.034528447 +0000 UTC m=+462.786151511" watchObservedRunningTime="2026-07-06 21:29:15.035290383 +0000 UTC m=+462.786913446" Jul 06 21:29:26.020837 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:26.020806 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" Jul 06 21:29:26.456081 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:26.456047 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h77bg"] Jul 06 21:29:26.456271 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:26.456228 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" podUID="fa8df390-7638-45f6-bdc4-9923cd9ae371" containerName="limitador" containerID="cri-o://34d3d25ef43d71757e32f6663d809daad7fc0cd8fca6d31151e38b1d67e5ec4f" gracePeriod=30 Jul 06 21:29:27.000223 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.000200 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" Jul 06 21:29:27.061935 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.061901 2573 generic.go:356] "Generic (PLEG): container finished" podID="fa8df390-7638-45f6-bdc4-9923cd9ae371" containerID="34d3d25ef43d71757e32f6663d809daad7fc0cd8fca6d31151e38b1d67e5ec4f" exitCode=0 Jul 06 21:29:27.062324 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.061962 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" Jul 06 21:29:27.062324 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.061965 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" event={"ID":"fa8df390-7638-45f6-bdc4-9923cd9ae371","Type":"ContainerDied","Data":"34d3d25ef43d71757e32f6663d809daad7fc0cd8fca6d31151e38b1d67e5ec4f"} Jul 06 21:29:27.062324 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.062019 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-h77bg" event={"ID":"fa8df390-7638-45f6-bdc4-9923cd9ae371","Type":"ContainerDied","Data":"0215128288c65336c0151986114a01a09f9b030ea58bb2283a172e39f2f1b762"} Jul 06 21:29:27.062324 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.062043 2573 scope.go:117] "RemoveContainer" containerID="34d3d25ef43d71757e32f6663d809daad7fc0cd8fca6d31151e38b1d67e5ec4f" Jul 06 21:29:27.069697 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.069678 2573 scope.go:117] "RemoveContainer" containerID="34d3d25ef43d71757e32f6663d809daad7fc0cd8fca6d31151e38b1d67e5ec4f" Jul 06 21:29:27.069962 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:29:27.069925 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34d3d25ef43d71757e32f6663d809daad7fc0cd8fca6d31151e38b1d67e5ec4f\": container with ID starting with 34d3d25ef43d71757e32f6663d809daad7fc0cd8fca6d31151e38b1d67e5ec4f not found: ID does not exist" containerID="34d3d25ef43d71757e32f6663d809daad7fc0cd8fca6d31151e38b1d67e5ec4f" Jul 06 21:29:27.070037 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.069969 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34d3d25ef43d71757e32f6663d809daad7fc0cd8fca6d31151e38b1d67e5ec4f"} err="failed to get container status \"34d3d25ef43d71757e32f6663d809daad7fc0cd8fca6d31151e38b1d67e5ec4f\": rpc error: code = NotFound desc = could not find container \"34d3d25ef43d71757e32f6663d809daad7fc0cd8fca6d31151e38b1d67e5ec4f\": container with ID starting with 34d3d25ef43d71757e32f6663d809daad7fc0cd8fca6d31151e38b1d67e5ec4f not found: ID does not exist" Jul 06 21:29:27.127373 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.127342 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkth6\" (UniqueName: \"kubernetes.io/projected/fa8df390-7638-45f6-bdc4-9923cd9ae371-kube-api-access-nkth6\") pod \"fa8df390-7638-45f6-bdc4-9923cd9ae371\" (UID: \"fa8df390-7638-45f6-bdc4-9923cd9ae371\") " Jul 06 21:29:27.127516 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.127392 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fa8df390-7638-45f6-bdc4-9923cd9ae371-config-file\") pod \"fa8df390-7638-45f6-bdc4-9923cd9ae371\" (UID: \"fa8df390-7638-45f6-bdc4-9923cd9ae371\") " Jul 06 21:29:27.127792 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.127761 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa8df390-7638-45f6-bdc4-9923cd9ae371-config-file" (OuterVolumeSpecName: "config-file") pod "fa8df390-7638-45f6-bdc4-9923cd9ae371" (UID: "fa8df390-7638-45f6-bdc4-9923cd9ae371"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:29:27.129926 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.129898 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa8df390-7638-45f6-bdc4-9923cd9ae371-kube-api-access-nkth6" (OuterVolumeSpecName: "kube-api-access-nkth6") pod "fa8df390-7638-45f6-bdc4-9923cd9ae371" (UID: "fa8df390-7638-45f6-bdc4-9923cd9ae371"). InnerVolumeSpecName "kube-api-access-nkth6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:29:27.228402 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.228369 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkth6\" (UniqueName: \"kubernetes.io/projected/fa8df390-7638-45f6-bdc4-9923cd9ae371-kube-api-access-nkth6\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:29:27.228402 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.228396 2573 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fa8df390-7638-45f6-bdc4-9923cd9ae371-config-file\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:29:27.386095 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.386066 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h77bg"] Jul 06 21:29:27.389768 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.389745 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h77bg"] Jul 06 21:29:27.997817 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.997781 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-hck88"] Jul 06 21:29:27.998103 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.998089 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa8df390-7638-45f6-bdc4-9923cd9ae371" containerName="limitador" Jul 06 21:29:27.998103 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.998103 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa8df390-7638-45f6-bdc4-9923cd9ae371" containerName="limitador" Jul 06 21:29:27.998225 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:27.998154 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa8df390-7638-45f6-bdc4-9923cd9ae371" containerName="limitador" Jul 06 21:29:28.002091 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:28.002070 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-hck88" Jul 06 21:29:28.004742 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:28.004723 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 21:29:28.004837 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:28.004725 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zpnlz\"" Jul 06 21:29:28.008707 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:28.008666 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-hck88"] Jul 06 21:29:28.023893 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:28.023873 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:28.033965 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:28.033932 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9c57\" (UniqueName: \"kubernetes.io/projected/7beb8af2-b284-4876-a6bf-47f7c9488b07-kube-api-access-b9c57\") pod \"postgres-69fb4594cf-hck88\" (UID: \"7beb8af2-b284-4876-a6bf-47f7c9488b07\") " pod="opendatahub/postgres-69fb4594cf-hck88" Jul 06 21:29:28.034062 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:28.033983 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7beb8af2-b284-4876-a6bf-47f7c9488b07-data\") pod \"postgres-69fb4594cf-hck88\" (UID: \"7beb8af2-b284-4876-a6bf-47f7c9488b07\") " pod="opendatahub/postgres-69fb4594cf-hck88" Jul 06 21:29:28.135346 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:28.135316 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9c57\" (UniqueName: \"kubernetes.io/projected/7beb8af2-b284-4876-a6bf-47f7c9488b07-kube-api-access-b9c57\") pod \"postgres-69fb4594cf-hck88\" (UID: \"7beb8af2-b284-4876-a6bf-47f7c9488b07\") " pod="opendatahub/postgres-69fb4594cf-hck88" Jul 06 21:29:28.135744 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:28.135359 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7beb8af2-b284-4876-a6bf-47f7c9488b07-data\") pod \"postgres-69fb4594cf-hck88\" (UID: \"7beb8af2-b284-4876-a6bf-47f7c9488b07\") " pod="opendatahub/postgres-69fb4594cf-hck88" Jul 06 21:29:28.135744 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:28.135674 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7beb8af2-b284-4876-a6bf-47f7c9488b07-data\") pod \"postgres-69fb4594cf-hck88\" (UID: \"7beb8af2-b284-4876-a6bf-47f7c9488b07\") " pod="opendatahub/postgres-69fb4594cf-hck88" Jul 06 21:29:28.143857 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:28.143825 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9c57\" (UniqueName: \"kubernetes.io/projected/7beb8af2-b284-4876-a6bf-47f7c9488b07-kube-api-access-b9c57\") pod \"postgres-69fb4594cf-hck88\" (UID: \"7beb8af2-b284-4876-a6bf-47f7c9488b07\") " pod="opendatahub/postgres-69fb4594cf-hck88" Jul 06 21:29:28.217168 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:28.217135 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa8df390-7638-45f6-bdc4-9923cd9ae371" path="/var/lib/kubelet/pods/fa8df390-7638-45f6-bdc4-9923cd9ae371/volumes" Jul 06 21:29:28.312282 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:28.312254 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-hck88" Jul 06 21:29:28.436334 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:28.436301 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-hck88"] Jul 06 21:29:28.439979 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:29:28.439933 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7beb8af2_b284_4876_a6bf_47f7c9488b07.slice/crio-39652b112cc9d835fd53dd697676aa90c3303709105bab2218d0de2fa7ffab80 WatchSource:0}: Error finding container 39652b112cc9d835fd53dd697676aa90c3303709105bab2218d0de2fa7ffab80: Status 404 returned error can't find the container with id 39652b112cc9d835fd53dd697676aa90c3303709105bab2218d0de2fa7ffab80 Jul 06 21:29:29.071346 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:29.071307 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-hck88" event={"ID":"7beb8af2-b284-4876-a6bf-47f7c9488b07","Type":"ContainerStarted","Data":"39652b112cc9d835fd53dd697676aa90c3303709105bab2218d0de2fa7ffab80"} Jul 06 21:29:43.570732 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:43.570710 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 21:29:44.131223 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:44.131181 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-hck88" event={"ID":"7beb8af2-b284-4876-a6bf-47f7c9488b07","Type":"ContainerStarted","Data":"63503d4c16c19140aacf272b90f70925064e4189ab58b67097b938337e7e5f17"} Jul 06 21:29:44.131433 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:44.131288 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-hck88" Jul 06 21:29:44.149014 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:44.148955 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-hck88" podStartSLOduration=2.022670883 podStartE2EDuration="17.148925483s" totalImagesPullingTime="15.1262546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:29:44.146835856 +0000 UTC m=+491.898458922" watchObservedRunningTime="2026-07-06 21:29:44.148925483 +0000 UTC m=+491.900548547" Jul 06 21:29:50.164838 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:50.164810 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-hck88" Jul 06 21:29:50.532653 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:50.532622 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:50.632820 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:50.632783 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:57.404808 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.404771 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:57.412846 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.410871 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-857948d5f4-lzt8n"] Jul 06 21:29:57.414726 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.414706 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-857948d5f4-lzt8n" Jul 06 21:29:57.417905 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.417885 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 21:29:57.417905 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.417898 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 21:29:57.418072 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.417933 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5lv5z\"" Jul 06 21:29:57.425321 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.425299 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-857948d5f4-lzt8n"] Jul 06 21:29:57.510718 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.510688 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:57.585045 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.585011 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/039b3fd0-25d4-4878-a405-4ce7721c5d71-webhook-cert\") pod \"maas-controller-857948d5f4-lzt8n\" (UID: \"039b3fd0-25d4-4878-a405-4ce7721c5d71\") " pod="opendatahub/maas-controller-857948d5f4-lzt8n" Jul 06 21:29:57.585165 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.585051 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsftn\" (UniqueName: \"kubernetes.io/projected/039b3fd0-25d4-4878-a405-4ce7721c5d71-kube-api-access-rsftn\") pod \"maas-controller-857948d5f4-lzt8n\" (UID: \"039b3fd0-25d4-4878-a405-4ce7721c5d71\") " pod="opendatahub/maas-controller-857948d5f4-lzt8n" Jul 06 21:29:57.600871 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.600832 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:57.629671 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.629644 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68768b879b-cjfqg"] Jul 06 21:29:57.632471 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.632456 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68768b879b-cjfqg" Jul 06 21:29:57.648072 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.648051 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68768b879b-cjfqg"] Jul 06 21:29:57.675989 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.675909 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:29:57.686090 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.686064 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/039b3fd0-25d4-4878-a405-4ce7721c5d71-webhook-cert\") pod \"maas-controller-857948d5f4-lzt8n\" (UID: \"039b3fd0-25d4-4878-a405-4ce7721c5d71\") " pod="opendatahub/maas-controller-857948d5f4-lzt8n" Jul 06 21:29:57.686180 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.686101 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsftn\" (UniqueName: \"kubernetes.io/projected/039b3fd0-25d4-4878-a405-4ce7721c5d71-kube-api-access-rsftn\") pod \"maas-controller-857948d5f4-lzt8n\" (UID: \"039b3fd0-25d4-4878-a405-4ce7721c5d71\") " pod="opendatahub/maas-controller-857948d5f4-lzt8n" Jul 06 21:29:57.688487 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.688469 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/039b3fd0-25d4-4878-a405-4ce7721c5d71-webhook-cert\") pod \"maas-controller-857948d5f4-lzt8n\" (UID: \"039b3fd0-25d4-4878-a405-4ce7721c5d71\") " pod="opendatahub/maas-controller-857948d5f4-lzt8n" Jul 06 21:29:57.695731 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.695706 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsftn\" (UniqueName: \"kubernetes.io/projected/039b3fd0-25d4-4878-a405-4ce7721c5d71-kube-api-access-rsftn\") pod \"maas-controller-857948d5f4-lzt8n\" (UID: \"039b3fd0-25d4-4878-a405-4ce7721c5d71\") " pod="opendatahub/maas-controller-857948d5f4-lzt8n" Jul 06 21:29:57.725588 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.725566 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-857948d5f4-lzt8n" Jul 06 21:29:57.787575 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.787531 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x6tq\" (UniqueName: \"kubernetes.io/projected/ffc19e29-55a7-42b1-a46f-9d1659feea91-kube-api-access-4x6tq\") pod \"maas-controller-68768b879b-cjfqg\" (UID: \"ffc19e29-55a7-42b1-a46f-9d1659feea91\") " pod="opendatahub/maas-controller-68768b879b-cjfqg" Jul 06 21:29:57.788181 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.788154 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ffc19e29-55a7-42b1-a46f-9d1659feea91-webhook-cert\") pod \"maas-controller-68768b879b-cjfqg\" (UID: \"ffc19e29-55a7-42b1-a46f-9d1659feea91\") " pod="opendatahub/maas-controller-68768b879b-cjfqg" Jul 06 21:29:57.850158 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.850129 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-857948d5f4-lzt8n"] Jul 06 21:29:57.853221 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:29:57.853194 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039b3fd0_25d4_4878_a405_4ce7721c5d71.slice/crio-93ee452456293ee0ec86dae6a5930704fca8af9392c3d5d7c12cad3d396a51ae WatchSource:0}: Error finding container 93ee452456293ee0ec86dae6a5930704fca8af9392c3d5d7c12cad3d396a51ae: Status 404 returned error can't find the container with id 93ee452456293ee0ec86dae6a5930704fca8af9392c3d5d7c12cad3d396a51ae Jul 06 21:29:57.888829 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.888799 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x6tq\" (UniqueName: \"kubernetes.io/projected/ffc19e29-55a7-42b1-a46f-9d1659feea91-kube-api-access-4x6tq\") pod \"maas-controller-68768b879b-cjfqg\" (UID: \"ffc19e29-55a7-42b1-a46f-9d1659feea91\") " pod="opendatahub/maas-controller-68768b879b-cjfqg" Jul 06 21:29:57.888933 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.888842 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ffc19e29-55a7-42b1-a46f-9d1659feea91-webhook-cert\") pod \"maas-controller-68768b879b-cjfqg\" (UID: \"ffc19e29-55a7-42b1-a46f-9d1659feea91\") " pod="opendatahub/maas-controller-68768b879b-cjfqg" Jul 06 21:29:57.891077 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.891060 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ffc19e29-55a7-42b1-a46f-9d1659feea91-webhook-cert\") pod \"maas-controller-68768b879b-cjfqg\" (UID: \"ffc19e29-55a7-42b1-a46f-9d1659feea91\") " pod="opendatahub/maas-controller-68768b879b-cjfqg" Jul 06 21:29:57.897159 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.897142 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x6tq\" (UniqueName: \"kubernetes.io/projected/ffc19e29-55a7-42b1-a46f-9d1659feea91-kube-api-access-4x6tq\") pod \"maas-controller-68768b879b-cjfqg\" (UID: \"ffc19e29-55a7-42b1-a46f-9d1659feea91\") " pod="opendatahub/maas-controller-68768b879b-cjfqg" Jul 06 21:29:57.942088 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:57.942030 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68768b879b-cjfqg" Jul 06 21:29:58.069398 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:58.069366 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68768b879b-cjfqg"] Jul 06 21:29:58.072424 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:29:58.072396 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc19e29_55a7_42b1_a46f_9d1659feea91.slice/crio-357acec92ca15875e9b496ae498b640d35329ef2d1807b15b18fa3d5441982f5 WatchSource:0}: Error finding container 357acec92ca15875e9b496ae498b640d35329ef2d1807b15b18fa3d5441982f5: Status 404 returned error can't find the container with id 357acec92ca15875e9b496ae498b640d35329ef2d1807b15b18fa3d5441982f5 Jul 06 21:29:58.187213 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:58.187171 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68768b879b-cjfqg" event={"ID":"ffc19e29-55a7-42b1-a46f-9d1659feea91","Type":"ContainerStarted","Data":"357acec92ca15875e9b496ae498b640d35329ef2d1807b15b18fa3d5441982f5"} Jul 06 21:29:58.188281 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:29:58.188253 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-857948d5f4-lzt8n" event={"ID":"039b3fd0-25d4-4878-a405-4ce7721c5d71","Type":"ContainerStarted","Data":"93ee452456293ee0ec86dae6a5930704fca8af9392c3d5d7c12cad3d396a51ae"} Jul 06 21:30:00.704026 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:00.703994 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:30:00.807843 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:00.807810 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:30:01.203320 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:01.203231 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68768b879b-cjfqg" event={"ID":"ffc19e29-55a7-42b1-a46f-9d1659feea91","Type":"ContainerStarted","Data":"a2529a210c962a493d69a082fe051e61c37e9683e77377babc163f4b4bf5a91c"} Jul 06 21:30:01.203320 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:01.203305 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68768b879b-cjfqg" Jul 06 21:30:01.204648 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:01.204627 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-857948d5f4-lzt8n" event={"ID":"039b3fd0-25d4-4878-a405-4ce7721c5d71","Type":"ContainerStarted","Data":"042ed3588b51ad3c92c1ac1f53fc646898448183230bbdf213ba488a9146f759"} Jul 06 21:30:01.204700 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:01.204676 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-857948d5f4-lzt8n" Jul 06 21:30:01.220825 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:01.220778 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68768b879b-cjfqg" podStartSLOduration=1.809288881 podStartE2EDuration="4.220763554s" totalImagesPullingTime="2.411474673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:29:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:30:01.220468932 +0000 UTC m=+508.972092008" watchObservedRunningTime="2026-07-06 21:30:01.220763554 +0000 UTC m=+508.972386618" Jul 06 21:30:01.242177 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:01.242122 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-857948d5f4-lzt8n" podStartSLOduration=1.623387152 podStartE2EDuration="4.242104401s" totalImagesPullingTime="2.618717249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:29:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:30:01.241932444 +0000 UTC m=+508.993555508" watchObservedRunningTime="2026-07-06 21:30:01.242104401 +0000 UTC m=+508.993727469" Jul 06 21:30:12.216745 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:12.216718 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-857948d5f4-lzt8n" Jul 06 21:30:12.217218 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:12.216775 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68768b879b-cjfqg" Jul 06 21:30:12.276676 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:12.276642 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-857948d5f4-lzt8n"] Jul 06 21:30:12.276896 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:12.276872 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-857948d5f4-lzt8n" podUID="039b3fd0-25d4-4878-a405-4ce7721c5d71" containerName="manager" containerID="cri-o://042ed3588b51ad3c92c1ac1f53fc646898448183230bbdf213ba488a9146f759" gracePeriod=30 Jul 06 21:30:12.524896 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:12.524870 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-857948d5f4-lzt8n" Jul 06 21:30:12.598570 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:12.598539 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/039b3fd0-25d4-4878-a405-4ce7721c5d71-webhook-cert\") pod \"039b3fd0-25d4-4878-a405-4ce7721c5d71\" (UID: \"039b3fd0-25d4-4878-a405-4ce7721c5d71\") " Jul 06 21:30:12.598570 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:12.598577 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsftn\" (UniqueName: \"kubernetes.io/projected/039b3fd0-25d4-4878-a405-4ce7721c5d71-kube-api-access-rsftn\") pod \"039b3fd0-25d4-4878-a405-4ce7721c5d71\" (UID: \"039b3fd0-25d4-4878-a405-4ce7721c5d71\") " Jul 06 21:30:12.600799 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:12.600774 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/039b3fd0-25d4-4878-a405-4ce7721c5d71-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "039b3fd0-25d4-4878-a405-4ce7721c5d71" (UID: "039b3fd0-25d4-4878-a405-4ce7721c5d71"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:30:12.600877 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:12.600834 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/039b3fd0-25d4-4878-a405-4ce7721c5d71-kube-api-access-rsftn" (OuterVolumeSpecName: "kube-api-access-rsftn") pod "039b3fd0-25d4-4878-a405-4ce7721c5d71" (UID: "039b3fd0-25d4-4878-a405-4ce7721c5d71"). InnerVolumeSpecName "kube-api-access-rsftn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:30:12.699639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:12.699603 2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/039b3fd0-25d4-4878-a405-4ce7721c5d71-webhook-cert\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:30:12.699639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:12.699633 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsftn\" (UniqueName: \"kubernetes.io/projected/039b3fd0-25d4-4878-a405-4ce7721c5d71-kube-api-access-rsftn\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:30:13.251140 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:13.251052 2573 generic.go:356] "Generic (PLEG): container finished" podID="039b3fd0-25d4-4878-a405-4ce7721c5d71" containerID="042ed3588b51ad3c92c1ac1f53fc646898448183230bbdf213ba488a9146f759" exitCode=0 Jul 06 21:30:13.251140 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:13.251101 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-857948d5f4-lzt8n" Jul 06 21:30:13.251648 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:13.251136 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-857948d5f4-lzt8n" event={"ID":"039b3fd0-25d4-4878-a405-4ce7721c5d71","Type":"ContainerDied","Data":"042ed3588b51ad3c92c1ac1f53fc646898448183230bbdf213ba488a9146f759"} Jul 06 21:30:13.251648 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:13.251184 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-857948d5f4-lzt8n" event={"ID":"039b3fd0-25d4-4878-a405-4ce7721c5d71","Type":"ContainerDied","Data":"93ee452456293ee0ec86dae6a5930704fca8af9392c3d5d7c12cad3d396a51ae"} Jul 06 21:30:13.251648 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:13.251206 2573 scope.go:117] "RemoveContainer" containerID="042ed3588b51ad3c92c1ac1f53fc646898448183230bbdf213ba488a9146f759" Jul 06 21:30:13.264897 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:13.264877 2573 scope.go:117] "RemoveContainer" containerID="042ed3588b51ad3c92c1ac1f53fc646898448183230bbdf213ba488a9146f759" Jul 06 21:30:13.265173 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:30:13.265153 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"042ed3588b51ad3c92c1ac1f53fc646898448183230bbdf213ba488a9146f759\": container with ID starting with 042ed3588b51ad3c92c1ac1f53fc646898448183230bbdf213ba488a9146f759 not found: ID does not exist" containerID="042ed3588b51ad3c92c1ac1f53fc646898448183230bbdf213ba488a9146f759" Jul 06 21:30:13.265250 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:13.265184 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042ed3588b51ad3c92c1ac1f53fc646898448183230bbdf213ba488a9146f759"} err="failed to get container status \"042ed3588b51ad3c92c1ac1f53fc646898448183230bbdf213ba488a9146f759\": rpc error: code = NotFound desc = could not find container \"042ed3588b51ad3c92c1ac1f53fc646898448183230bbdf213ba488a9146f759\": container with ID starting with 042ed3588b51ad3c92c1ac1f53fc646898448183230bbdf213ba488a9146f759 not found: ID does not exist" Jul 06 21:30:13.276545 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:13.276521 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-857948d5f4-lzt8n"] Jul 06 21:30:13.280559 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:13.280512 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-857948d5f4-lzt8n"] Jul 06 21:30:14.217468 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:14.217435 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="039b3fd0-25d4-4878-a405-4ce7721c5d71" path="/var/lib/kubelet/pods/039b3fd0-25d4-4878-a405-4ce7721c5d71/volumes" Jul 06 21:30:35.212873 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.212830 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:30:35.557305 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.557264 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:30:35.610480 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.610447 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5cb4b76d8d-m544q"] Jul 06 21:30:35.610763 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.610751 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="039b3fd0-25d4-4878-a405-4ce7721c5d71" containerName="manager" Jul 06 21:30:35.610805 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.610765 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="039b3fd0-25d4-4878-a405-4ce7721c5d71" containerName="manager" Jul 06 21:30:35.610842 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.610831 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="039b3fd0-25d4-4878-a405-4ce7721c5d71" containerName="manager" Jul 06 21:30:35.615546 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.615524 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5cb4b76d8d-m544q" Jul 06 21:30:35.618188 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.618164 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 21:30:35.618292 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.618236 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-cxwgs\"" Jul 06 21:30:35.633622 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.633599 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5cb4b76d8d-m544q"] Jul 06 21:30:35.688441 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.688403 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcsks\" (UniqueName: \"kubernetes.io/projected/e8108c29-6388-4310-a3a3-03883e97ec70-kube-api-access-kcsks\") pod \"maas-api-5cb4b76d8d-m544q\" (UID: \"e8108c29-6388-4310-a3a3-03883e97ec70\") " pod="opendatahub/maas-api-5cb4b76d8d-m544q" Jul 06 21:30:35.688441 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.688439 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8108c29-6388-4310-a3a3-03883e97ec70-maas-api-tls\") pod \"maas-api-5cb4b76d8d-m544q\" (UID: \"e8108c29-6388-4310-a3a3-03883e97ec70\") " pod="opendatahub/maas-api-5cb4b76d8d-m544q" Jul 06 21:30:35.688675 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.688521 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8108c29-6388-4310-a3a3-03883e97ec70-openshift-service-ca\") pod \"maas-api-5cb4b76d8d-m544q\" (UID: \"e8108c29-6388-4310-a3a3-03883e97ec70\") " pod="opendatahub/maas-api-5cb4b76d8d-m544q" Jul 06 21:30:35.789377 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.789338 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8108c29-6388-4310-a3a3-03883e97ec70-openshift-service-ca\") pod \"maas-api-5cb4b76d8d-m544q\" (UID: \"e8108c29-6388-4310-a3a3-03883e97ec70\") " pod="opendatahub/maas-api-5cb4b76d8d-m544q" Jul 06 21:30:35.789566 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.789393 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcsks\" (UniqueName: \"kubernetes.io/projected/e8108c29-6388-4310-a3a3-03883e97ec70-kube-api-access-kcsks\") pod \"maas-api-5cb4b76d8d-m544q\" (UID: \"e8108c29-6388-4310-a3a3-03883e97ec70\") " pod="opendatahub/maas-api-5cb4b76d8d-m544q" Jul 06 21:30:35.789566 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.789413 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8108c29-6388-4310-a3a3-03883e97ec70-maas-api-tls\") pod \"maas-api-5cb4b76d8d-m544q\" (UID: \"e8108c29-6388-4310-a3a3-03883e97ec70\") " pod="opendatahub/maas-api-5cb4b76d8d-m544q" Jul 06 21:30:35.789566 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:30:35.789508 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 06 21:30:35.789771 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:30:35.789574 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8108c29-6388-4310-a3a3-03883e97ec70-maas-api-tls podName:e8108c29-6388-4310-a3a3-03883e97ec70 nodeName:}" failed. No retries permitted until 2026-07-06 21:30:36.289556601 +0000 UTC m=+544.041179649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e8108c29-6388-4310-a3a3-03883e97ec70-maas-api-tls") pod "maas-api-5cb4b76d8d-m544q" (UID: "e8108c29-6388-4310-a3a3-03883e97ec70") : secret "maas-api-serving-cert" not found Jul 06 21:30:35.789985 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.789934 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8108c29-6388-4310-a3a3-03883e97ec70-openshift-service-ca\") pod \"maas-api-5cb4b76d8d-m544q\" (UID: \"e8108c29-6388-4310-a3a3-03883e97ec70\") " pod="opendatahub/maas-api-5cb4b76d8d-m544q" Jul 06 21:30:35.798775 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:35.798748 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcsks\" (UniqueName: \"kubernetes.io/projected/e8108c29-6388-4310-a3a3-03883e97ec70-kube-api-access-kcsks\") pod \"maas-api-5cb4b76d8d-m544q\" (UID: \"e8108c29-6388-4310-a3a3-03883e97ec70\") " pod="opendatahub/maas-api-5cb4b76d8d-m544q" Jul 06 21:30:36.006304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.006224 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:30:36.099986 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.099955 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:30:36.129218 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.129190 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6kkvm"] Jul 06 21:30:36.131570 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.131555 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-6kkvm" Jul 06 21:30:36.134593 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.134574 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 21:30:36.134852 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.134832 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-sgnnv\"" Jul 06 21:30:36.143311 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.143292 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6kkvm"] Jul 06 21:30:36.192882 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.192854 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h5lf\" (UniqueName: \"kubernetes.io/projected/174f6560-4c7f-4f6a-bdde-ee3c86efcf81-kube-api-access-5h5lf\") pod \"payload-processing-58f86c7798-6kkvm\" (UID: \"174f6560-4c7f-4f6a-bdde-ee3c86efcf81\") " pod="openshift-ingress/payload-processing-58f86c7798-6kkvm" Jul 06 21:30:36.193062 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.192914 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/174f6560-4c7f-4f6a-bdde-ee3c86efcf81-plugins-config-volume\") pod \"payload-processing-58f86c7798-6kkvm\" (UID: \"174f6560-4c7f-4f6a-bdde-ee3c86efcf81\") " pod="openshift-ingress/payload-processing-58f86c7798-6kkvm" Jul 06 21:30:36.290139 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.290091 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:30:36.293919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.293897 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8108c29-6388-4310-a3a3-03883e97ec70-maas-api-tls\") pod \"maas-api-5cb4b76d8d-m544q\" (UID: \"e8108c29-6388-4310-a3a3-03883e97ec70\") " pod="opendatahub/maas-api-5cb4b76d8d-m544q" Jul 06 21:30:36.294048 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.293930 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h5lf\" (UniqueName: \"kubernetes.io/projected/174f6560-4c7f-4f6a-bdde-ee3c86efcf81-kube-api-access-5h5lf\") pod \"payload-processing-58f86c7798-6kkvm\" (UID: \"174f6560-4c7f-4f6a-bdde-ee3c86efcf81\") " pod="openshift-ingress/payload-processing-58f86c7798-6kkvm" Jul 06 21:30:36.294048 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.293985 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/174f6560-4c7f-4f6a-bdde-ee3c86efcf81-plugins-config-volume\") pod \"payload-processing-58f86c7798-6kkvm\" (UID: \"174f6560-4c7f-4f6a-bdde-ee3c86efcf81\") " pod="openshift-ingress/payload-processing-58f86c7798-6kkvm" Jul 06 21:30:36.294619 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.294601 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/174f6560-4c7f-4f6a-bdde-ee3c86efcf81-plugins-config-volume\") pod \"payload-processing-58f86c7798-6kkvm\" (UID: \"174f6560-4c7f-4f6a-bdde-ee3c86efcf81\") " pod="openshift-ingress/payload-processing-58f86c7798-6kkvm" Jul 06 21:30:36.296336 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.296320 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8108c29-6388-4310-a3a3-03883e97ec70-maas-api-tls\") pod \"maas-api-5cb4b76d8d-m544q\" (UID: \"e8108c29-6388-4310-a3a3-03883e97ec70\") " pod="opendatahub/maas-api-5cb4b76d8d-m544q" Jul 06 21:30:36.322939 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.322909 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h5lf\" (UniqueName: \"kubernetes.io/projected/174f6560-4c7f-4f6a-bdde-ee3c86efcf81-kube-api-access-5h5lf\") pod \"payload-processing-58f86c7798-6kkvm\" (UID: \"174f6560-4c7f-4f6a-bdde-ee3c86efcf81\") " pod="openshift-ingress/payload-processing-58f86c7798-6kkvm" Jul 06 21:30:36.442308 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.442263 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-6kkvm" Jul 06 21:30:36.525392 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.525284 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5cb4b76d8d-m544q" Jul 06 21:30:36.574575 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.574359 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6kkvm"] Jul 06 21:30:36.579056 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:30:36.579027 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod174f6560_4c7f_4f6a_bdde_ee3c86efcf81.slice/crio-57edf35654d3407ff1698944a3259871077be26dc410fb459d5b49e9f5e008e7 WatchSource:0}: Error finding container 57edf35654d3407ff1698944a3259871077be26dc410fb459d5b49e9f5e008e7: Status 404 returned error can't find the container with id 57edf35654d3407ff1698944a3259871077be26dc410fb459d5b49e9f5e008e7 Jul 06 21:30:36.675878 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:36.675851 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5cb4b76d8d-m544q"] Jul 06 21:30:36.678417 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:30:36.678391 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8108c29_6388_4310_a3a3_03883e97ec70.slice/crio-389d9773c2eb8b552f85c7b0a94be462ecd6ad519d4fe5de48cf91eb147738cb WatchSource:0}: Error finding container 389d9773c2eb8b552f85c7b0a94be462ecd6ad519d4fe5de48cf91eb147738cb: Status 404 returned error can't find the container with id 389d9773c2eb8b552f85c7b0a94be462ecd6ad519d4fe5de48cf91eb147738cb Jul 06 21:30:37.344113 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:37.344069 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5cb4b76d8d-m544q" event={"ID":"e8108c29-6388-4310-a3a3-03883e97ec70","Type":"ContainerStarted","Data":"389d9773c2eb8b552f85c7b0a94be462ecd6ad519d4fe5de48cf91eb147738cb"} Jul 06 21:30:37.345469 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:37.345412 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-6kkvm" event={"ID":"174f6560-4c7f-4f6a-bdde-ee3c86efcf81","Type":"ContainerStarted","Data":"57edf35654d3407ff1698944a3259871077be26dc410fb459d5b49e9f5e008e7"} Jul 06 21:30:38.351289 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:38.350977 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5cb4b76d8d-m544q" event={"ID":"e8108c29-6388-4310-a3a3-03883e97ec70","Type":"ContainerStarted","Data":"e869e48d87d4ada644847e27b7a6b22f3edb2a559abaf51476ebbceacfa0fcad"} Jul 06 21:30:38.351289 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:38.351155 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5cb4b76d8d-m544q" Jul 06 21:30:38.352639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:38.352617 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-6kkvm" event={"ID":"174f6560-4c7f-4f6a-bdde-ee3c86efcf81","Type":"ContainerStarted","Data":"a9ce93c08a2b54b6a70953d0e89c952b0255931c091da53887f14b4f0a6d3ded"} Jul 06 21:30:38.352762 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:38.352748 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-6kkvm" Jul 06 21:30:38.369684 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:38.369638 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-5cb4b76d8d-m544q" podStartSLOduration=1.775308764 podStartE2EDuration="3.369622922s" totalImagesPullingTime="1.594314158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:30:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:30:38.369011097 +0000 UTC m=+546.120634162" watchObservedRunningTime="2026-07-06 21:30:38.369622922 +0000 UTC m=+546.121245985" Jul 06 21:30:38.385660 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:38.385609 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-6kkvm" podStartSLOduration=0.712925245 podStartE2EDuration="2.385590553s" totalImagesPullingTime="1.672665308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:30:38.385590329 +0000 UTC m=+546.137213414" watchObservedRunningTime="2026-07-06 21:30:38.385590553 +0000 UTC m=+546.137213620" Jul 06 21:30:44.362629 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:44.362592 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5cb4b76d8d-m544q" Jul 06 21:30:45.456218 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:45.456183 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:30:45.546793 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:45.546758 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:30:49.359627 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:30:49.359597 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-6kkvm" Jul 06 21:31:04.148121 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:04.148089 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:31:11.998002 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:11.997969 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz"] Jul 06 21:31:12.003163 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.003138 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.006000 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.005974 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 21:31:12.007347 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.007326 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-b5mbx\"" Jul 06 21:31:12.007444 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.007378 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 21:31:12.007444 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.007430 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 06 21:31:12.013044 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.013015 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz"] Jul 06 21:31:12.030834 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.030809 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:31:12.075611 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.075577 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fbc398e-ba13-4d93-ac4b-a38340c367d4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.075742 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.075612 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fchg9\" (UniqueName: \"kubernetes.io/projected/2fbc398e-ba13-4d93-ac4b-a38340c367d4-kube-api-access-fchg9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.075742 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.075645 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fbc398e-ba13-4d93-ac4b-a38340c367d4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.075742 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.075684 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fbc398e-ba13-4d93-ac4b-a38340c367d4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.075879 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.075748 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbc398e-ba13-4d93-ac4b-a38340c367d4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.075879 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.075771 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fbc398e-ba13-4d93-ac4b-a38340c367d4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.176740 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.176710 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbc398e-ba13-4d93-ac4b-a38340c367d4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.176915 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.176750 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fbc398e-ba13-4d93-ac4b-a38340c367d4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.176915 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.176804 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fbc398e-ba13-4d93-ac4b-a38340c367d4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.176915 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.176827 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fchg9\" (UniqueName: \"kubernetes.io/projected/2fbc398e-ba13-4d93-ac4b-a38340c367d4-kube-api-access-fchg9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.176915 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.176859 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fbc398e-ba13-4d93-ac4b-a38340c367d4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.176915 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.176883 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fbc398e-ba13-4d93-ac4b-a38340c367d4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.177298 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.177268 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fbc398e-ba13-4d93-ac4b-a38340c367d4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.177413 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.177297 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fbc398e-ba13-4d93-ac4b-a38340c367d4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.177413 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.177333 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fbc398e-ba13-4d93-ac4b-a38340c367d4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.179169 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.179149 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fbc398e-ba13-4d93-ac4b-a38340c367d4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.179372 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.179354 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fbc398e-ba13-4d93-ac4b-a38340c367d4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.185647 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.185623 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fchg9\" (UniqueName: \"kubernetes.io/projected/2fbc398e-ba13-4d93-ac4b-a38340c367d4-kube-api-access-fchg9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dzwgz\" (UID: \"2fbc398e-ba13-4d93-ac4b-a38340c367d4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.314426 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.314395 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:12.443059 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.443033 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz"] Jul 06 21:31:12.445148 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:31:12.445117 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fbc398e_ba13_4d93_ac4b_a38340c367d4.slice/crio-cd9ea564ccdf19b02bd54c16735b6da5dda89c7a16aba7cc96a714aef8e840b1 WatchSource:0}: Error finding container cd9ea564ccdf19b02bd54c16735b6da5dda89c7a16aba7cc96a714aef8e840b1: Status 404 returned error can't find the container with id cd9ea564ccdf19b02bd54c16735b6da5dda89c7a16aba7cc96a714aef8e840b1 Jul 06 21:31:12.498351 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:12.498315 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" event={"ID":"2fbc398e-ba13-4d93-ac4b-a38340c367d4","Type":"ContainerStarted","Data":"cd9ea564ccdf19b02bd54c16735b6da5dda89c7a16aba7cc96a714aef8e840b1"} Jul 06 21:31:14.508538 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:14.508503 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" event={"ID":"2fbc398e-ba13-4d93-ac4b-a38340c367d4","Type":"ContainerStarted","Data":"67b0b78d8b821efbd35212f1d64e25b268b52b31b206019b06376c144b41dbbb"} Jul 06 21:31:14.526642 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:14.526595 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" podStartSLOduration=2.52859129 podStartE2EDuration="3.526579411s" totalImagesPullingTime="997.988121ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:31:14.525805237 +0000 UTC m=+582.277428302" watchObservedRunningTime="2026-07-06 21:31:14.526579411 +0000 UTC m=+582.278202476" Jul 06 21:31:20.398105 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.398068 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb"] Jul 06 21:31:20.401426 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.401407 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.403888 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.403870 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 21:31:20.410619 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.410558 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb"] Jul 06 21:31:20.449519 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.449488 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ed3c382-62ca-4267-8133-4b5d9971f446-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.449678 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.449524 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ed3c382-62ca-4267-8133-4b5d9971f446-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.449678 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.449551 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ed3c382-62ca-4267-8133-4b5d9971f446-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.449678 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.449632 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ed3c382-62ca-4267-8133-4b5d9971f446-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.449835 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.449691 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-487f2\" (UniqueName: \"kubernetes.io/projected/2ed3c382-62ca-4267-8133-4b5d9971f446-kube-api-access-487f2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.449835 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.449742 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ed3c382-62ca-4267-8133-4b5d9971f446-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.506680 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.506648 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:31:20.550560 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.550526 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ed3c382-62ca-4267-8133-4b5d9971f446-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.550699 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.550564 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ed3c382-62ca-4267-8133-4b5d9971f446-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.550699 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.550599 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-487f2\" (UniqueName: \"kubernetes.io/projected/2ed3c382-62ca-4267-8133-4b5d9971f446-kube-api-access-487f2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.550699 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.550643 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ed3c382-62ca-4267-8133-4b5d9971f446-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.550699 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.550696 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ed3c382-62ca-4267-8133-4b5d9971f446-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.550904 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.550724 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ed3c382-62ca-4267-8133-4b5d9971f446-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.551025 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.551002 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ed3c382-62ca-4267-8133-4b5d9971f446-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.551108 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.551077 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ed3c382-62ca-4267-8133-4b5d9971f446-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.551108 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.551098 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ed3c382-62ca-4267-8133-4b5d9971f446-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.552846 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.552829 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ed3c382-62ca-4267-8133-4b5d9971f446-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.553135 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.553116 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ed3c382-62ca-4267-8133-4b5d9971f446-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.558379 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.558358 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-487f2\" (UniqueName: \"kubernetes.io/projected/2ed3c382-62ca-4267-8133-4b5d9971f446-kube-api-access-487f2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xqnmb\" (UID: \"2ed3c382-62ca-4267-8133-4b5d9971f446\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.711801 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.711726 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:20.838282 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:20.838258 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb"] Jul 06 21:31:20.840426 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:31:20.840401 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed3c382_62ca_4267_8133_4b5d9971f446.slice/crio-947e5e49c3e2136e5926e3082e576a440658e3f0c00f6c9fcd36612776b80631 WatchSource:0}: Error finding container 947e5e49c3e2136e5926e3082e576a440658e3f0c00f6c9fcd36612776b80631: Status 404 returned error can't find the container with id 947e5e49c3e2136e5926e3082e576a440658e3f0c00f6c9fcd36612776b80631 Jul 06 21:31:21.543372 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:21.543334 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" event={"ID":"2ed3c382-62ca-4267-8133-4b5d9971f446","Type":"ContainerStarted","Data":"54b8e757effae96162a42b7652252ad898cfeeec4447f0c586e6047855bf41fc"} Jul 06 21:31:21.543858 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:21.543392 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" event={"ID":"2ed3c382-62ca-4267-8133-4b5d9971f446","Type":"ContainerStarted","Data":"947e5e49c3e2136e5926e3082e576a440658e3f0c00f6c9fcd36612776b80631"} Jul 06 21:31:21.563546 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:21.563495 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" podStartSLOduration=1.386412308 podStartE2EDuration="1.563481058s" totalImagesPullingTime="177.06875ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:31:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:31:21.561473571 +0000 UTC m=+589.313096635" watchObservedRunningTime="2026-07-06 21:31:21.563481058 +0000 UTC m=+589.315104121" Jul 06 21:31:22.315541 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:22.315508 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:22.315720 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:22.315574 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:22.327668 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:22.327645 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:22.558469 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:22.558441 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dzwgz" Jul 06 21:31:30.712134 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:30.712098 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:30.712527 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:30.712150 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:30.724525 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:30.724499 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:31.593255 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:31.593228 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xqnmb" Jul 06 21:31:34.089824 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:34.089796 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:31:34.090288 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:34.090013 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:31:47.188855 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:47.188820 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:31:51.845145 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:31:51.845094 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:00.344845 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:32:00.344746 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:24.776821 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:32:24.776786 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:27.772667 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:32:27.772623 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:35.318395 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:32:35.318352 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:37.836577 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:32:37.836520 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:37.897120 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:32:37.897089 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:46.422384 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:32:46.422350 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:57.383122 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:32:57.383069 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:57.478103 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:32:57.478074 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:58.587336 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:32:58.587276 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:32:58.681263 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:32:58.681231 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:06.412409 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:33:06.412372 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:33:06.518155 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:33:06.518119 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:09.064634 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:09.064589 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:09.163619 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:09.163586 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68768b879b-cjfqg"] Jul 06 21:34:09.163828 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:09.163804 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-68768b879b-cjfqg" podUID="ffc19e29-55a7-42b1-a46f-9d1659feea91" containerName="manager" containerID="cri-o://a2529a210c962a493d69a082fe051e61c37e9683e77377babc163f4b4bf5a91c" gracePeriod=30 Jul 06 21:34:09.409393 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:09.409372 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68768b879b-cjfqg" Jul 06 21:34:09.430270 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:09.430245 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4x6tq\" (UniqueName: \"kubernetes.io/projected/ffc19e29-55a7-42b1-a46f-9d1659feea91-kube-api-access-4x6tq\") pod \"ffc19e29-55a7-42b1-a46f-9d1659feea91\" (UID: \"ffc19e29-55a7-42b1-a46f-9d1659feea91\") " Jul 06 21:34:09.430403 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:09.430282 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ffc19e29-55a7-42b1-a46f-9d1659feea91-webhook-cert\") pod \"ffc19e29-55a7-42b1-a46f-9d1659feea91\" (UID: \"ffc19e29-55a7-42b1-a46f-9d1659feea91\") " Jul 06 21:34:09.432161 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:09.432133 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffc19e29-55a7-42b1-a46f-9d1659feea91-kube-api-access-4x6tq" (OuterVolumeSpecName: "kube-api-access-4x6tq") pod "ffc19e29-55a7-42b1-a46f-9d1659feea91" (UID: "ffc19e29-55a7-42b1-a46f-9d1659feea91"). InnerVolumeSpecName "kube-api-access-4x6tq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:34:09.432274 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:09.432238 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffc19e29-55a7-42b1-a46f-9d1659feea91-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ffc19e29-55a7-42b1-a46f-9d1659feea91" (UID: "ffc19e29-55a7-42b1-a46f-9d1659feea91"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:34:09.531212 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:09.531180 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4x6tq\" (UniqueName: \"kubernetes.io/projected/ffc19e29-55a7-42b1-a46f-9d1659feea91-kube-api-access-4x6tq\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:34:09.531212 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:09.531207 2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ffc19e29-55a7-42b1-a46f-9d1659feea91-webhook-cert\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:34:10.220488 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:10.220455 2573 generic.go:356] "Generic (PLEG): container finished" podID="ffc19e29-55a7-42b1-a46f-9d1659feea91" containerID="a2529a210c962a493d69a082fe051e61c37e9683e77377babc163f4b4bf5a91c" exitCode=0 Jul 06 21:34:10.220904 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:10.220513 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68768b879b-cjfqg" event={"ID":"ffc19e29-55a7-42b1-a46f-9d1659feea91","Type":"ContainerDied","Data":"a2529a210c962a493d69a082fe051e61c37e9683e77377babc163f4b4bf5a91c"} Jul 06 21:34:10.220904 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:10.220516 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68768b879b-cjfqg" Jul 06 21:34:10.220904 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:10.220545 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68768b879b-cjfqg" event={"ID":"ffc19e29-55a7-42b1-a46f-9d1659feea91","Type":"ContainerDied","Data":"357acec92ca15875e9b496ae498b640d35329ef2d1807b15b18fa3d5441982f5"} Jul 06 21:34:10.220904 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:10.220565 2573 scope.go:117] "RemoveContainer" containerID="a2529a210c962a493d69a082fe051e61c37e9683e77377babc163f4b4bf5a91c" Jul 06 21:34:10.229181 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:10.229164 2573 scope.go:117] "RemoveContainer" containerID="a2529a210c962a493d69a082fe051e61c37e9683e77377babc163f4b4bf5a91c" Jul 06 21:34:10.229434 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:34:10.229413 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2529a210c962a493d69a082fe051e61c37e9683e77377babc163f4b4bf5a91c\": container with ID starting with a2529a210c962a493d69a082fe051e61c37e9683e77377babc163f4b4bf5a91c not found: ID does not exist" containerID="a2529a210c962a493d69a082fe051e61c37e9683e77377babc163f4b4bf5a91c" Jul 06 21:34:10.229525 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:10.229440 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2529a210c962a493d69a082fe051e61c37e9683e77377babc163f4b4bf5a91c"} err="failed to get container status \"a2529a210c962a493d69a082fe051e61c37e9683e77377babc163f4b4bf5a91c\": rpc error: code = NotFound desc = could not find container \"a2529a210c962a493d69a082fe051e61c37e9683e77377babc163f4b4bf5a91c\": container with ID starting with a2529a210c962a493d69a082fe051e61c37e9683e77377babc163f4b4bf5a91c not found: ID does not exist" Jul 06 21:34:10.241350 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:10.241330 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68768b879b-cjfqg"] Jul 06 21:34:10.247663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:10.247644 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68768b879b-cjfqg"] Jul 06 21:34:10.923713 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:10.923677 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:11.039897 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.039866 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68768b879b-q4vms"] Jul 06 21:34:11.040241 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.040227 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffc19e29-55a7-42b1-a46f-9d1659feea91" containerName="manager" Jul 06 21:34:11.040284 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.040242 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffc19e29-55a7-42b1-a46f-9d1659feea91" containerName="manager" Jul 06 21:34:11.040318 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.040291 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffc19e29-55a7-42b1-a46f-9d1659feea91" containerName="manager" Jul 06 21:34:11.045097 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.045072 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68768b879b-q4vms" Jul 06 21:34:11.049995 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.049975 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 21:34:11.050125 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.049974 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5lv5z\"" Jul 06 21:34:11.050125 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.049977 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 21:34:11.052700 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.052675 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68768b879b-q4vms"] Jul 06 21:34:11.145762 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.145733 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84db4fec-45f0-4192-aedc-c572329b4210-webhook-cert\") pod \"maas-controller-68768b879b-q4vms\" (UID: \"84db4fec-45f0-4192-aedc-c572329b4210\") " pod="opendatahub/maas-controller-68768b879b-q4vms" Jul 06 21:34:11.145932 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.145793 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4bsc\" (UniqueName: \"kubernetes.io/projected/84db4fec-45f0-4192-aedc-c572329b4210-kube-api-access-h4bsc\") pod \"maas-controller-68768b879b-q4vms\" (UID: \"84db4fec-45f0-4192-aedc-c572329b4210\") " pod="opendatahub/maas-controller-68768b879b-q4vms" Jul 06 21:34:11.246433 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.246360 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84db4fec-45f0-4192-aedc-c572329b4210-webhook-cert\") pod \"maas-controller-68768b879b-q4vms\" (UID: \"84db4fec-45f0-4192-aedc-c572329b4210\") " pod="opendatahub/maas-controller-68768b879b-q4vms" Jul 06 21:34:11.246778 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.246525 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4bsc\" (UniqueName: \"kubernetes.io/projected/84db4fec-45f0-4192-aedc-c572329b4210-kube-api-access-h4bsc\") pod \"maas-controller-68768b879b-q4vms\" (UID: \"84db4fec-45f0-4192-aedc-c572329b4210\") " pod="opendatahub/maas-controller-68768b879b-q4vms" Jul 06 21:34:11.248719 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.248700 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84db4fec-45f0-4192-aedc-c572329b4210-webhook-cert\") pod \"maas-controller-68768b879b-q4vms\" (UID: \"84db4fec-45f0-4192-aedc-c572329b4210\") " pod="opendatahub/maas-controller-68768b879b-q4vms" Jul 06 21:34:11.255436 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.255416 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4bsc\" (UniqueName: \"kubernetes.io/projected/84db4fec-45f0-4192-aedc-c572329b4210-kube-api-access-h4bsc\") pod \"maas-controller-68768b879b-q4vms\" (UID: \"84db4fec-45f0-4192-aedc-c572329b4210\") " pod="opendatahub/maas-controller-68768b879b-q4vms" Jul 06 21:34:11.356706 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.356679 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68768b879b-q4vms" Jul 06 21:34:11.480820 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.480795 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68768b879b-q4vms"] Jul 06 21:34:11.483072 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:34:11.483046 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84db4fec_45f0_4192_aedc_c572329b4210.slice/crio-9dea6e7fc5dea0b71cb82996a24baf93b9dc303336f40ee9106571af1d909138 WatchSource:0}: Error finding container 9dea6e7fc5dea0b71cb82996a24baf93b9dc303336f40ee9106571af1d909138: Status 404 returned error can't find the container with id 9dea6e7fc5dea0b71cb82996a24baf93b9dc303336f40ee9106571af1d909138 Jul 06 21:34:11.484235 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:11.484216 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 21:34:12.217400 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:12.217369 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffc19e29-55a7-42b1-a46f-9d1659feea91" path="/var/lib/kubelet/pods/ffc19e29-55a7-42b1-a46f-9d1659feea91/volumes" Jul 06 21:34:12.231358 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:12.231325 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68768b879b-q4vms" event={"ID":"84db4fec-45f0-4192-aedc-c572329b4210","Type":"ContainerStarted","Data":"d331753ddb4b30ca043ba651f0fb960e062397783de3575a9963710b9ca3fea8"} Jul 06 21:34:12.231490 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:12.231372 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68768b879b-q4vms" event={"ID":"84db4fec-45f0-4192-aedc-c572329b4210","Type":"ContainerStarted","Data":"9dea6e7fc5dea0b71cb82996a24baf93b9dc303336f40ee9106571af1d909138"} Jul 06 21:34:12.231490 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:12.231429 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68768b879b-q4vms" Jul 06 21:34:12.249915 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:12.249833 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68768b879b-q4vms" podStartSLOduration=0.736182818 podStartE2EDuration="1.24981969s" totalImagesPullingTime="513.636872ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:34:12.24819987 +0000 UTC m=+759.999822934" watchObservedRunningTime="2026-07-06 21:34:12.24981969 +0000 UTC m=+760.001442755" Jul 06 21:34:23.240182 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:23.240155 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68768b879b-q4vms" Jul 06 21:34:51.650348 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:51.650310 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:51.742842 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:51.742804 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:51.859176 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:51.859141 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:57.751854 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:57.751762 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:57.858733 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:57.858703 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:57.969665 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:57.969628 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:34:58.130119 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:34:58.130088 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:35:03.966386 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:35:03.966356 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:35:04.045122 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:35:04.045085 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:36:06.274162 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:36:06.274132 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:36:34.112960 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:36:34.112873 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:36:34.114013 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:36:34.113991 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:36:44.288333 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:36:44.288302 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:36:44.403884 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:36:44.403849 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:38:07.362064 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:38:07.362028 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:41:34.142519 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:41:34.142489 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:41:34.144022 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:41:34.144002 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:42:12.148625 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:42:12.148590 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:42:13.400861 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:42:13.400828 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:45:42.868530 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:45:42.868451 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:45:48.242927 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:45:48.242896 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:45:56.524966 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:45:56.524917 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:01.700825 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:01.700785 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:04.645334 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:04.645300 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:04.740338 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:04.740303 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:05.113252 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:05.113216 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:11.894048 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:11.894007 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:11.963347 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:11.963312 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:12.365845 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.365812 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:12.536886 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.536853 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh"] Jul 06 21:46:12.539847 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.539831 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" Jul 06 21:46:12.543715 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.543693 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-6b851c3b-serving-cert\"" Jul 06 21:46:12.555105 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.555081 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh"] Jul 06 21:46:12.604473 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.604441 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:12.656006 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.655900 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-maas-api-tls\") pod \"maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh\" (UID: \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\") " pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" Jul 06 21:46:12.656006 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.655936 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-openshift-service-ca\") pod \"maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh\" (UID: \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\") " pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" Jul 06 21:46:12.656190 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.656034 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppk6f\" (UniqueName: \"kubernetes.io/projected/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-kube-api-access-ppk6f\") pod \"maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh\" (UID: \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\") " pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" Jul 06 21:46:12.693231 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.693199 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:12.756584 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.756545 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppk6f\" (UniqueName: \"kubernetes.io/projected/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-kube-api-access-ppk6f\") pod \"maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh\" (UID: \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\") " pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" Jul 06 21:46:12.756781 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.756633 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-maas-api-tls\") pod \"maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh\" (UID: \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\") " pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" Jul 06 21:46:12.756781 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.756661 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-openshift-service-ca\") pod \"maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh\" (UID: \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\") " pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" Jul 06 21:46:12.757341 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.757315 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-openshift-service-ca\") pod \"maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh\" (UID: \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\") " pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" Jul 06 21:46:12.759063 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.759043 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-maas-api-tls\") pod \"maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh\" (UID: \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\") " pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" Jul 06 21:46:12.767807 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.767787 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppk6f\" (UniqueName: \"kubernetes.io/projected/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-kube-api-access-ppk6f\") pod \"maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh\" (UID: \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\") " pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" Jul 06 21:46:12.850102 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.850058 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" Jul 06 21:46:12.987072 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.987042 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh"] Jul 06 21:46:12.990637 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:46:12.990592 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaacde6bd_676f_4dfe_ae8f_047fb33a1ea9.slice/crio-a69eeda02b7472db629a8d3c4d0301c24d9aeaf43c8872ea688c1a9d9afca557 WatchSource:0}: Error finding container a69eeda02b7472db629a8d3c4d0301c24d9aeaf43c8872ea688c1a9d9afca557: Status 404 returned error can't find the container with id a69eeda02b7472db629a8d3c4d0301c24d9aeaf43c8872ea688c1a9d9afca557 Jul 06 21:46:12.992124 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:12.992105 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 21:46:13.033369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:13.033341 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:13.132355 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:13.132326 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:13.191708 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:13.191629 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" event={"ID":"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9","Type":"ContainerStarted","Data":"a69eeda02b7472db629a8d3c4d0301c24d9aeaf43c8872ea688c1a9d9afca557"} Jul 06 21:46:14.198118 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:14.198090 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" event={"ID":"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9","Type":"ContainerStarted","Data":"f3d1bf9e61562a64b430905db950c80c9841ca48e5d737be91707d24200c85b0"} Jul 06 21:46:14.198605 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:14.198225 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" Jul 06 21:46:14.216164 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:14.216107 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" podStartSLOduration=1.779027824 podStartE2EDuration="2.216087419s" totalImagesPullingTime="437.059595ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:46:14.214806288 +0000 UTC m=+1481.966429372" watchObservedRunningTime="2026-07-06 21:46:14.216087419 +0000 UTC m=+1481.967710483" Jul 06 21:46:14.672095 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:14.672061 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:17.481192 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:17.481153 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:17.668749 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:17.668713 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:17.782465 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:17.782351 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:17.783167 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:17.783141 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh"] Jul 06 21:46:17.783826 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:17.783368 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" podUID="aacde6bd-676f-4dfe-ae8f-047fb33a1ea9" containerName="maas-api" containerID="cri-o://f3d1bf9e61562a64b430905db950c80c9841ca48e5d737be91707d24200c85b0" gracePeriod=30 Jul 06 21:46:17.880407 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:17.880377 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:18.028005 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.027982 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" Jul 06 21:46:18.206622 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.206537 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppk6f\" (UniqueName: \"kubernetes.io/projected/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-kube-api-access-ppk6f\") pod \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\" (UID: \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\") " Jul 06 21:46:18.206622 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.206596 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-openshift-service-ca\") pod \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\" (UID: \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\") " Jul 06 21:46:18.206806 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.206641 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-maas-api-tls\") pod \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\" (UID: \"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9\") " Jul 06 21:46:18.206994 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.206963 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aacde6bd-676f-4dfe-ae8f-047fb33a1ea9" (UID: "aacde6bd-676f-4dfe-ae8f-047fb33a1ea9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:46:18.208600 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.208572 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-kube-api-access-ppk6f" (OuterVolumeSpecName: "kube-api-access-ppk6f") pod "aacde6bd-676f-4dfe-ae8f-047fb33a1ea9" (UID: "aacde6bd-676f-4dfe-ae8f-047fb33a1ea9"). InnerVolumeSpecName "kube-api-access-ppk6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:46:18.208735 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.208711 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aacde6bd-676f-4dfe-ae8f-047fb33a1ea9" (UID: "aacde6bd-676f-4dfe-ae8f-047fb33a1ea9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:46:18.218154 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.218127 2573 generic.go:356] "Generic (PLEG): container finished" podID="aacde6bd-676f-4dfe-ae8f-047fb33a1ea9" containerID="f3d1bf9e61562a64b430905db950c80c9841ca48e5d737be91707d24200c85b0" exitCode=0 Jul 06 21:46:18.218255 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.218180 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" Jul 06 21:46:18.218255 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.218212 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" event={"ID":"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9","Type":"ContainerDied","Data":"f3d1bf9e61562a64b430905db950c80c9841ca48e5d737be91707d24200c85b0"} Jul 06 21:46:18.218358 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.218267 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh" event={"ID":"aacde6bd-676f-4dfe-ae8f-047fb33a1ea9","Type":"ContainerDied","Data":"a69eeda02b7472db629a8d3c4d0301c24d9aeaf43c8872ea688c1a9d9afca557"} Jul 06 21:46:18.218358 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.218310 2573 scope.go:117] "RemoveContainer" containerID="f3d1bf9e61562a64b430905db950c80c9841ca48e5d737be91707d24200c85b0" Jul 06 21:46:18.226991 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.226967 2573 scope.go:117] "RemoveContainer" containerID="f3d1bf9e61562a64b430905db950c80c9841ca48e5d737be91707d24200c85b0" Jul 06 21:46:18.227394 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:46:18.227369 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3d1bf9e61562a64b430905db950c80c9841ca48e5d737be91707d24200c85b0\": container with ID starting with f3d1bf9e61562a64b430905db950c80c9841ca48e5d737be91707d24200c85b0 not found: ID does not exist" containerID="f3d1bf9e61562a64b430905db950c80c9841ca48e5d737be91707d24200c85b0" Jul 06 21:46:18.227460 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.227403 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3d1bf9e61562a64b430905db950c80c9841ca48e5d737be91707d24200c85b0"} err="failed to get container status \"f3d1bf9e61562a64b430905db950c80c9841ca48e5d737be91707d24200c85b0\": rpc error: code = NotFound desc = could not find container \"f3d1bf9e61562a64b430905db950c80c9841ca48e5d737be91707d24200c85b0\": container with ID starting with f3d1bf9e61562a64b430905db950c80c9841ca48e5d737be91707d24200c85b0 not found: ID does not exist" Jul 06 21:46:18.241497 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.241473 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh"] Jul 06 21:46:18.246250 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.246230 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-6b851c3b-574845bd87-9jwrh"] Jul 06 21:46:18.307710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.307685 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-openshift-service-ca\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:46:18.307710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.307707 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-maas-api-tls\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:46:18.307855 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:18.307719 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppk6f\" (UniqueName: \"kubernetes.io/projected/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9-kube-api-access-ppk6f\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:46:20.217714 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:20.217679 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aacde6bd-676f-4dfe-ae8f-047fb33a1ea9" path="/var/lib/kubelet/pods/aacde6bd-676f-4dfe-ae8f-047fb33a1ea9/volumes" Jul 06 21:46:21.128413 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:21.128376 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:23.546764 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:23.546726 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:23.612473 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:23.612444 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:23.842878 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:23.842807 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:23.942821 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:23.942789 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:25.020364 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.020329 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:25.091066 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.091035 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66"] Jul 06 21:46:25.091519 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.091504 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aacde6bd-676f-4dfe-ae8f-047fb33a1ea9" containerName="maas-api" Jul 06 21:46:25.091562 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.091524 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="aacde6bd-676f-4dfe-ae8f-047fb33a1ea9" containerName="maas-api" Jul 06 21:46:25.091615 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.091604 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="aacde6bd-676f-4dfe-ae8f-047fb33a1ea9" containerName="maas-api" Jul 06 21:46:25.096052 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.096030 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:25.099144 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.099123 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-7e20a6c0-serving-cert\"" Jul 06 21:46:25.107715 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.107694 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66"] Jul 06 21:46:25.115333 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.115312 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:25.191379 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.191350 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:25.263711 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.263679 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d983ea9-0297-4b27-ad94-1051b3e13c97-openshift-service-ca\") pod \"maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66\" (UID: \"4d983ea9-0297-4b27-ad94-1051b3e13c97\") " pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:25.263862 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.263726 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gl5d\" (UniqueName: \"kubernetes.io/projected/4d983ea9-0297-4b27-ad94-1051b3e13c97-kube-api-access-8gl5d\") pod \"maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66\" (UID: \"4d983ea9-0297-4b27-ad94-1051b3e13c97\") " pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:25.263862 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.263766 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d983ea9-0297-4b27-ad94-1051b3e13c97-maas-api-tls\") pod \"maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66\" (UID: \"4d983ea9-0297-4b27-ad94-1051b3e13c97\") " pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:25.296257 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.296235 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:25.364680 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.364651 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d983ea9-0297-4b27-ad94-1051b3e13c97-openshift-service-ca\") pod \"maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66\" (UID: \"4d983ea9-0297-4b27-ad94-1051b3e13c97\") " pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:25.364840 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.364705 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gl5d\" (UniqueName: \"kubernetes.io/projected/4d983ea9-0297-4b27-ad94-1051b3e13c97-kube-api-access-8gl5d\") pod \"maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66\" (UID: \"4d983ea9-0297-4b27-ad94-1051b3e13c97\") " pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:25.364840 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.364768 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d983ea9-0297-4b27-ad94-1051b3e13c97-maas-api-tls\") pod \"maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66\" (UID: \"4d983ea9-0297-4b27-ad94-1051b3e13c97\") " pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:25.365312 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.365293 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d983ea9-0297-4b27-ad94-1051b3e13c97-openshift-service-ca\") pod \"maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66\" (UID: \"4d983ea9-0297-4b27-ad94-1051b3e13c97\") " pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:25.367448 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.367429 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d983ea9-0297-4b27-ad94-1051b3e13c97-maas-api-tls\") pod \"maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66\" (UID: \"4d983ea9-0297-4b27-ad94-1051b3e13c97\") " pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:25.375143 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.375124 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gl5d\" (UniqueName: \"kubernetes.io/projected/4d983ea9-0297-4b27-ad94-1051b3e13c97-kube-api-access-8gl5d\") pod \"maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66\" (UID: \"4d983ea9-0297-4b27-ad94-1051b3e13c97\") " pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:25.406000 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.405981 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:25.538609 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.538571 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66"] Jul 06 21:46:25.543682 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:46:25.543651 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d983ea9_0297_4b27_ad94_1051b3e13c97.slice/crio-468bdf7c08284689a8ed73bd7f5072871b04d700564e33a29f649df440d004a7 WatchSource:0}: Error finding container 468bdf7c08284689a8ed73bd7f5072871b04d700564e33a29f649df440d004a7: Status 404 returned error can't find the container with id 468bdf7c08284689a8ed73bd7f5072871b04d700564e33a29f649df440d004a7 Jul 06 21:46:25.597257 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.597227 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:25.698121 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:25.698086 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:26.253451 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:26.253370 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" event={"ID":"4d983ea9-0297-4b27-ad94-1051b3e13c97","Type":"ContainerStarted","Data":"c7af4df33756a9b482995794b422865dbc08548ca16f43dda22b7af18b274edf"} Jul 06 21:46:26.253451 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:26.253421 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" event={"ID":"4d983ea9-0297-4b27-ad94-1051b3e13c97","Type":"ContainerStarted","Data":"468bdf7c08284689a8ed73bd7f5072871b04d700564e33a29f649df440d004a7"} Jul 06 21:46:26.253868 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:26.253518 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:26.275174 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:26.275128 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" podStartSLOduration=0.908936451 podStartE2EDuration="1.275113174s" totalImagesPullingTime="366.176723ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:46:26.273724018 +0000 UTC m=+1494.025347083" watchObservedRunningTime="2026-07-06 21:46:26.275113174 +0000 UTC m=+1494.026736237" Jul 06 21:46:26.762527 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:26.762493 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:29.507475 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:29.507442 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:30.106339 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.106312 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:30.396976 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.396876 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66"] Jul 06 21:46:30.397142 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.397088 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" podUID="4d983ea9-0297-4b27-ad94-1051b3e13c97" containerName="maas-api" containerID="cri-o://c7af4df33756a9b482995794b422865dbc08548ca16f43dda22b7af18b274edf" gracePeriod=30 Jul 06 21:46:30.402625 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.402604 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:30.426116 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.426088 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:30.641089 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.641061 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:30.705809 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.705732 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d983ea9-0297-4b27-ad94-1051b3e13c97-openshift-service-ca\") pod \"4d983ea9-0297-4b27-ad94-1051b3e13c97\" (UID: \"4d983ea9-0297-4b27-ad94-1051b3e13c97\") " Jul 06 21:46:30.705809 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.705774 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gl5d\" (UniqueName: \"kubernetes.io/projected/4d983ea9-0297-4b27-ad94-1051b3e13c97-kube-api-access-8gl5d\") pod \"4d983ea9-0297-4b27-ad94-1051b3e13c97\" (UID: \"4d983ea9-0297-4b27-ad94-1051b3e13c97\") " Jul 06 21:46:30.706043 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.705871 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d983ea9-0297-4b27-ad94-1051b3e13c97-maas-api-tls\") pod \"4d983ea9-0297-4b27-ad94-1051b3e13c97\" (UID: \"4d983ea9-0297-4b27-ad94-1051b3e13c97\") " Jul 06 21:46:30.706102 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.706066 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d983ea9-0297-4b27-ad94-1051b3e13c97-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4d983ea9-0297-4b27-ad94-1051b3e13c97" (UID: "4d983ea9-0297-4b27-ad94-1051b3e13c97"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:46:30.707762 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.707741 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d983ea9-0297-4b27-ad94-1051b3e13c97-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4d983ea9-0297-4b27-ad94-1051b3e13c97" (UID: "4d983ea9-0297-4b27-ad94-1051b3e13c97"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:46:30.707862 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.707773 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d983ea9-0297-4b27-ad94-1051b3e13c97-kube-api-access-8gl5d" (OuterVolumeSpecName: "kube-api-access-8gl5d") pod "4d983ea9-0297-4b27-ad94-1051b3e13c97" (UID: "4d983ea9-0297-4b27-ad94-1051b3e13c97"). InnerVolumeSpecName "kube-api-access-8gl5d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:46:30.806589 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.806554 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d983ea9-0297-4b27-ad94-1051b3e13c97-maas-api-tls\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:46:30.806589 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.806588 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d983ea9-0297-4b27-ad94-1051b3e13c97-openshift-service-ca\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:46:30.806589 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:30.806598 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gl5d\" (UniqueName: \"kubernetes.io/projected/4d983ea9-0297-4b27-ad94-1051b3e13c97-kube-api-access-8gl5d\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:46:31.274413 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:31.274380 2573 generic.go:356] "Generic (PLEG): container finished" podID="4d983ea9-0297-4b27-ad94-1051b3e13c97" containerID="c7af4df33756a9b482995794b422865dbc08548ca16f43dda22b7af18b274edf" exitCode=0 Jul 06 21:46:31.274617 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:31.274419 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" Jul 06 21:46:31.274617 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:31.274467 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" event={"ID":"4d983ea9-0297-4b27-ad94-1051b3e13c97","Type":"ContainerDied","Data":"c7af4df33756a9b482995794b422865dbc08548ca16f43dda22b7af18b274edf"} Jul 06 21:46:31.274617 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:31.274524 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66" event={"ID":"4d983ea9-0297-4b27-ad94-1051b3e13c97","Type":"ContainerDied","Data":"468bdf7c08284689a8ed73bd7f5072871b04d700564e33a29f649df440d004a7"} Jul 06 21:46:31.274617 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:31.274556 2573 scope.go:117] "RemoveContainer" containerID="c7af4df33756a9b482995794b422865dbc08548ca16f43dda22b7af18b274edf" Jul 06 21:46:31.283094 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:31.283077 2573 scope.go:117] "RemoveContainer" containerID="c7af4df33756a9b482995794b422865dbc08548ca16f43dda22b7af18b274edf" Jul 06 21:46:31.283331 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:46:31.283312 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7af4df33756a9b482995794b422865dbc08548ca16f43dda22b7af18b274edf\": container with ID starting with c7af4df33756a9b482995794b422865dbc08548ca16f43dda22b7af18b274edf not found: ID does not exist" containerID="c7af4df33756a9b482995794b422865dbc08548ca16f43dda22b7af18b274edf" Jul 06 21:46:31.283374 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:31.283340 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7af4df33756a9b482995794b422865dbc08548ca16f43dda22b7af18b274edf"} err="failed to get container status \"c7af4df33756a9b482995794b422865dbc08548ca16f43dda22b7af18b274edf\": rpc error: code = NotFound desc = could not find container \"c7af4df33756a9b482995794b422865dbc08548ca16f43dda22b7af18b274edf\": container with ID starting with c7af4df33756a9b482995794b422865dbc08548ca16f43dda22b7af18b274edf not found: ID does not exist" Jul 06 21:46:31.295524 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:31.295503 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66"] Jul 06 21:46:31.299445 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:31.299423 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-7e20a6c0-567b6fb959-txr66"] Jul 06 21:46:32.217848 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:32.217811 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d983ea9-0297-4b27-ad94-1051b3e13c97" path="/var/lib/kubelet/pods/4d983ea9-0297-4b27-ad94-1051b3e13c97/volumes" Jul 06 21:46:34.167116 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:34.167084 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:46:34.168246 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:34.168226 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:46:36.238032 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:36.237929 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:36.338392 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:36.338360 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:37.728507 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:37.728474 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:42.252324 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:42.252287 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:42.320908 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:42.320873 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:42.584052 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:42.584020 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:42.668047 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:42.668018 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:43.682926 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:43.682880 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:43.783074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:43.783041 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:43.900382 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:43.900346 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt"] Jul 06 21:46:43.900726 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:43.900708 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d983ea9-0297-4b27-ad94-1051b3e13c97" containerName="maas-api" Jul 06 21:46:43.900807 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:43.900730 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d983ea9-0297-4b27-ad94-1051b3e13c97" containerName="maas-api" Jul 06 21:46:43.900860 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:43.900813 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d983ea9-0297-4b27-ad94-1051b3e13c97" containerName="maas-api" Jul 06 21:46:43.905800 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:43.905779 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:43.908448 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:43.908431 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-c14d9971-serving-cert\"" Jul 06 21:46:43.917506 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:43.917485 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt"] Jul 06 21:46:43.949083 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:43.949027 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:43.998902 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:43.998872 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7z68\" (UniqueName: \"kubernetes.io/projected/10156cbc-b399-4340-8bb9-7f49aadc87b4-kube-api-access-v7z68\") pod \"maas-api-e2e-derive-c14d9971-86dcd97454-76zxt\" (UID: \"10156cbc-b399-4340-8bb9-7f49aadc87b4\") " pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:43.999048 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:43.998915 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10156cbc-b399-4340-8bb9-7f49aadc87b4-openshift-service-ca\") pod \"maas-api-e2e-derive-c14d9971-86dcd97454-76zxt\" (UID: \"10156cbc-b399-4340-8bb9-7f49aadc87b4\") " pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:43.999048 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:43.999010 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10156cbc-b399-4340-8bb9-7f49aadc87b4-maas-api-tls\") pod \"maas-api-e2e-derive-c14d9971-86dcd97454-76zxt\" (UID: \"10156cbc-b399-4340-8bb9-7f49aadc87b4\") " pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:44.100413 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:44.100378 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7z68\" (UniqueName: \"kubernetes.io/projected/10156cbc-b399-4340-8bb9-7f49aadc87b4-kube-api-access-v7z68\") pod \"maas-api-e2e-derive-c14d9971-86dcd97454-76zxt\" (UID: \"10156cbc-b399-4340-8bb9-7f49aadc87b4\") " pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:44.100577 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:44.100427 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10156cbc-b399-4340-8bb9-7f49aadc87b4-openshift-service-ca\") pod \"maas-api-e2e-derive-c14d9971-86dcd97454-76zxt\" (UID: \"10156cbc-b399-4340-8bb9-7f49aadc87b4\") " pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:44.100577 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:44.100469 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10156cbc-b399-4340-8bb9-7f49aadc87b4-maas-api-tls\") pod \"maas-api-e2e-derive-c14d9971-86dcd97454-76zxt\" (UID: \"10156cbc-b399-4340-8bb9-7f49aadc87b4\") " pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:44.100727 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:46:44.100650 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-c14d9971-serving-cert: secret "maas-api-e2e-derive-c14d9971-serving-cert" not found Jul 06 21:46:44.100727 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:46:44.100719 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10156cbc-b399-4340-8bb9-7f49aadc87b4-maas-api-tls podName:10156cbc-b399-4340-8bb9-7f49aadc87b4 nodeName:}" failed. No retries permitted until 2026-07-06 21:46:44.600698193 +0000 UTC m=+1512.352321245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/10156cbc-b399-4340-8bb9-7f49aadc87b4-maas-api-tls") pod "maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" (UID: "10156cbc-b399-4340-8bb9-7f49aadc87b4") : secret "maas-api-e2e-derive-c14d9971-serving-cert" not found Jul 06 21:46:44.101157 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:44.101132 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10156cbc-b399-4340-8bb9-7f49aadc87b4-openshift-service-ca\") pod \"maas-api-e2e-derive-c14d9971-86dcd97454-76zxt\" (UID: \"10156cbc-b399-4340-8bb9-7f49aadc87b4\") " pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:44.111685 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:44.111658 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7z68\" (UniqueName: \"kubernetes.io/projected/10156cbc-b399-4340-8bb9-7f49aadc87b4-kube-api-access-v7z68\") pod \"maas-api-e2e-derive-c14d9971-86dcd97454-76zxt\" (UID: \"10156cbc-b399-4340-8bb9-7f49aadc87b4\") " pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:44.359812 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:44.359772 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:44.482824 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:44.482785 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:44.586729 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:44.586688 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:44.605853 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:44.605825 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10156cbc-b399-4340-8bb9-7f49aadc87b4-maas-api-tls\") pod \"maas-api-e2e-derive-c14d9971-86dcd97454-76zxt\" (UID: \"10156cbc-b399-4340-8bb9-7f49aadc87b4\") " pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:44.608171 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:44.608153 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10156cbc-b399-4340-8bb9-7f49aadc87b4-maas-api-tls\") pod \"maas-api-e2e-derive-c14d9971-86dcd97454-76zxt\" (UID: \"10156cbc-b399-4340-8bb9-7f49aadc87b4\") " pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:44.815349 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:44.815316 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:44.956376 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:44.956339 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt"] Jul 06 21:46:44.959993 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:46:44.959935 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10156cbc_b399_4340_8bb9_7f49aadc87b4.slice/crio-d0c63245ca17e5da44a0d1e471f9345edf6f6689941d98281afb96530c04268f WatchSource:0}: Error finding container d0c63245ca17e5da44a0d1e471f9345edf6f6689941d98281afb96530c04268f: Status 404 returned error can't find the container with id d0c63245ca17e5da44a0d1e471f9345edf6f6689941d98281afb96530c04268f Jul 06 21:46:45.333213 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:45.333179 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" event={"ID":"10156cbc-b399-4340-8bb9-7f49aadc87b4","Type":"ContainerStarted","Data":"d0c63245ca17e5da44a0d1e471f9345edf6f6689941d98281afb96530c04268f"} Jul 06 21:46:45.648235 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:45.648193 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:46.339594 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:46.339561 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" event={"ID":"10156cbc-b399-4340-8bb9-7f49aadc87b4","Type":"ContainerStarted","Data":"9ee65129f46290ddb27f31993bb22fc71355ddbd384eb6e46e5a3429b01c87a3"} Jul 06 21:46:46.340005 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:46.339645 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:46.357129 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:46.357083 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" podStartSLOduration=2.932758604 podStartE2EDuration="3.357069351s" totalImagesPullingTime="424.310747ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:46:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:46:46.356245095 +0000 UTC m=+1514.107868160" watchObservedRunningTime="2026-07-06 21:46:46.357069351 +0000 UTC m=+1514.108692415" Jul 06 21:46:48.066936 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:48.066897 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:48.305154 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:48.305122 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:48.405022 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:48.404913 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:48.595369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:48.595337 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt"] Jul 06 21:46:48.595531 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:48.595506 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" podUID="10156cbc-b399-4340-8bb9-7f49aadc87b4" containerName="maas-api" containerID="cri-o://9ee65129f46290ddb27f31993bb22fc71355ddbd384eb6e46e5a3429b01c87a3" gracePeriod=30 Jul 06 21:46:48.612665 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:48.612639 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:48.845610 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:48.845580 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:48.938335 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:48.938300 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10156cbc-b399-4340-8bb9-7f49aadc87b4-openshift-service-ca\") pod \"10156cbc-b399-4340-8bb9-7f49aadc87b4\" (UID: \"10156cbc-b399-4340-8bb9-7f49aadc87b4\") " Jul 06 21:46:48.938490 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:48.938358 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7z68\" (UniqueName: \"kubernetes.io/projected/10156cbc-b399-4340-8bb9-7f49aadc87b4-kube-api-access-v7z68\") pod \"10156cbc-b399-4340-8bb9-7f49aadc87b4\" (UID: \"10156cbc-b399-4340-8bb9-7f49aadc87b4\") " Jul 06 21:46:48.938490 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:48.938408 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10156cbc-b399-4340-8bb9-7f49aadc87b4-maas-api-tls\") pod \"10156cbc-b399-4340-8bb9-7f49aadc87b4\" (UID: \"10156cbc-b399-4340-8bb9-7f49aadc87b4\") " Jul 06 21:46:48.938685 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:48.938665 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10156cbc-b399-4340-8bb9-7f49aadc87b4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10156cbc-b399-4340-8bb9-7f49aadc87b4" (UID: "10156cbc-b399-4340-8bb9-7f49aadc87b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:46:48.940428 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:48.940401 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10156cbc-b399-4340-8bb9-7f49aadc87b4-kube-api-access-v7z68" (OuterVolumeSpecName: "kube-api-access-v7z68") pod "10156cbc-b399-4340-8bb9-7f49aadc87b4" (UID: "10156cbc-b399-4340-8bb9-7f49aadc87b4"). InnerVolumeSpecName "kube-api-access-v7z68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:46:48.940498 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:48.940419 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10156cbc-b399-4340-8bb9-7f49aadc87b4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10156cbc-b399-4340-8bb9-7f49aadc87b4" (UID: "10156cbc-b399-4340-8bb9-7f49aadc87b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:46:49.038995 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:49.038965 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7z68\" (UniqueName: \"kubernetes.io/projected/10156cbc-b399-4340-8bb9-7f49aadc87b4-kube-api-access-v7z68\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:46:49.038995 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:49.038994 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10156cbc-b399-4340-8bb9-7f49aadc87b4-maas-api-tls\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:46:49.039159 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:49.039006 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10156cbc-b399-4340-8bb9-7f49aadc87b4-openshift-service-ca\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:46:49.356003 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:49.355910 2573 generic.go:356] "Generic (PLEG): container finished" podID="10156cbc-b399-4340-8bb9-7f49aadc87b4" containerID="9ee65129f46290ddb27f31993bb22fc71355ddbd384eb6e46e5a3429b01c87a3" exitCode=0 Jul 06 21:46:49.356003 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:49.355963 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" event={"ID":"10156cbc-b399-4340-8bb9-7f49aadc87b4","Type":"ContainerDied","Data":"9ee65129f46290ddb27f31993bb22fc71355ddbd384eb6e46e5a3429b01c87a3"} Jul 06 21:46:49.356003 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:49.356001 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" event={"ID":"10156cbc-b399-4340-8bb9-7f49aadc87b4","Type":"ContainerDied","Data":"d0c63245ca17e5da44a0d1e471f9345edf6f6689941d98281afb96530c04268f"} Jul 06 21:46:49.356494 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:49.356021 2573 scope.go:117] "RemoveContainer" containerID="9ee65129f46290ddb27f31993bb22fc71355ddbd384eb6e46e5a3429b01c87a3" Jul 06 21:46:49.356494 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:49.355964 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt" Jul 06 21:46:49.364665 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:49.364646 2573 scope.go:117] "RemoveContainer" containerID="9ee65129f46290ddb27f31993bb22fc71355ddbd384eb6e46e5a3429b01c87a3" Jul 06 21:46:49.364916 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:46:49.364897 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ee65129f46290ddb27f31993bb22fc71355ddbd384eb6e46e5a3429b01c87a3\": container with ID starting with 9ee65129f46290ddb27f31993bb22fc71355ddbd384eb6e46e5a3429b01c87a3 not found: ID does not exist" containerID="9ee65129f46290ddb27f31993bb22fc71355ddbd384eb6e46e5a3429b01c87a3" Jul 06 21:46:49.364985 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:49.364925 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ee65129f46290ddb27f31993bb22fc71355ddbd384eb6e46e5a3429b01c87a3"} err="failed to get container status \"9ee65129f46290ddb27f31993bb22fc71355ddbd384eb6e46e5a3429b01c87a3\": rpc error: code = NotFound desc = could not find container \"9ee65129f46290ddb27f31993bb22fc71355ddbd384eb6e46e5a3429b01c87a3\": container with ID starting with 9ee65129f46290ddb27f31993bb22fc71355ddbd384eb6e46e5a3429b01c87a3 not found: ID does not exist" Jul 06 21:46:49.386698 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:49.386669 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt"] Jul 06 21:46:49.395097 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:49.395070 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-c14d9971-86dcd97454-76zxt"] Jul 06 21:46:50.153492 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:50.153439 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:50.217828 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:50.217797 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10156cbc-b399-4340-8bb9-7f49aadc87b4" path="/var/lib/kubelet/pods/10156cbc-b399-4340-8bb9-7f49aadc87b4/volumes" Jul 06 21:46:54.293757 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:54.293647 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:54.375204 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:54.375171 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:54.805220 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:54.805181 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:54.907595 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:54.907560 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:55.003270 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:55.003239 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:46:56.455786 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:46:56.455754 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:02.401825 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:47:02.401792 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:02.508602 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:47:02.508560 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:02.603755 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:47:02.603715 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:02.702452 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:47:02.702369 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:03.150210 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:47:03.150160 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:29.144884 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:47:29.144847 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:34.935114 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:47:34.935065 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:35.059849 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:47:35.059817 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:35.153091 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:47:35.153059 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:51.125692 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:47:51.125655 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:57.154962 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:47:57.154899 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:57.264030 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:47:57.263992 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:47:57.363691 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:47:57.363660 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:07.942887 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:07.942850 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:08.814781 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:08.814749 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:14.975153 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:14.975116 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:15.088208 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:15.088176 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:15.222725 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:15.222698 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:15.325749 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:15.325716 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:16.124451 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:16.124419 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:16.224846 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:16.224811 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:16.323604 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:16.323577 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:17.994538 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:17.994504 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:23.834344 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:23.834309 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:23.901512 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:23.901489 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:24.571101 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:24.571073 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:30.724883 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:30.724847 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:30.804803 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:30.804768 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:31.006922 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:31.006840 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:31.345916 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:31.345881 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:31.642332 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:31.642242 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:34.400743 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.400704 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:34.458068 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.458037 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7"] Jul 06 21:48:34.458417 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.458402 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10156cbc-b399-4340-8bb9-7f49aadc87b4" containerName="maas-api" Jul 06 21:48:34.458501 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.458421 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="10156cbc-b399-4340-8bb9-7f49aadc87b4" containerName="maas-api" Jul 06 21:48:34.458560 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.458521 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="10156cbc-b399-4340-8bb9-7f49aadc87b4" containerName="maas-api" Jul 06 21:48:34.461282 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.461257 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:34.464081 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.464055 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-9f64f2fa-serving-cert\"" Jul 06 21:48:34.481296 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.481269 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7"] Jul 06 21:48:34.548848 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.548810 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:34.604129 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.604096 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-maas-api-tls\") pod \"maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7\" (UID: \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\") " pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:34.604293 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.604140 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-openshift-service-ca\") pod \"maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7\" (UID: \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\") " pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:34.604293 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.604269 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttcvd\" (UniqueName: \"kubernetes.io/projected/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-kube-api-access-ttcvd\") pod \"maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7\" (UID: \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\") " pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:34.669594 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.669514 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:34.704734 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.704689 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttcvd\" (UniqueName: \"kubernetes.io/projected/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-kube-api-access-ttcvd\") pod \"maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7\" (UID: \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\") " pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:34.704907 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.704750 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-maas-api-tls\") pod \"maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7\" (UID: \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\") " pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:34.704907 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.704792 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-openshift-service-ca\") pod \"maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7\" (UID: \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\") " pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:34.705573 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.705550 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-openshift-service-ca\") pod \"maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7\" (UID: \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\") " pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:34.707229 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.707203 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-maas-api-tls\") pod \"maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7\" (UID: \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\") " pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:34.713156 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.713134 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttcvd\" (UniqueName: \"kubernetes.io/projected/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-kube-api-access-ttcvd\") pod \"maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7\" (UID: \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\") " pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:34.772153 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.772119 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:34.910068 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:34.910032 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7"] Jul 06 21:48:34.913376 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:48:34.913350 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36bdb4ef_945f_40dc_a1af_aa715ae2e1e1.slice/crio-715350cf6cfb6b3d20017283757628a63040f842029c1a1ca140d81d1cc69527 WatchSource:0}: Error finding container 715350cf6cfb6b3d20017283757628a63040f842029c1a1ca140d81d1cc69527: Status 404 returned error can't find the container with id 715350cf6cfb6b3d20017283757628a63040f842029c1a1ca140d81d1cc69527 Jul 06 21:48:35.071703 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:35.071669 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:35.172969 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:35.172907 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:35.806312 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:35.806274 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" event={"ID":"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1","Type":"ContainerStarted","Data":"b7c464b4cfc971ccefcf31b1e7bd6671de604e3cb5042f1886b1174ca21c3ab5"} Jul 06 21:48:35.806732 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:35.806330 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" event={"ID":"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1","Type":"ContainerStarted","Data":"715350cf6cfb6b3d20017283757628a63040f842029c1a1ca140d81d1cc69527"} Jul 06 21:48:35.806732 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:35.806378 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:35.825648 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:35.825585 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" podStartSLOduration=1.40844318 podStartE2EDuration="1.825566083s" totalImagesPullingTime="417.122903ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:48:35.825375932 +0000 UTC m=+1623.576998998" watchObservedRunningTime="2026-07-06 21:48:35.825566083 +0000 UTC m=+1623.577189149" Jul 06 21:48:38.925650 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:38.925619 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:39.655351 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:39.655307 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:39.759179 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:39.759148 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:40.117427 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:40.117397 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:41.816416 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:41.816385 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:45.906223 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:45.906188 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:46.007896 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:46.007859 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:46.313819 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:46.313792 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:46.628885 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:46.628808 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7"] Jul 06 21:48:46.629136 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:46.629058 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" podUID="36bdb4ef-945f-40dc-a1af-aa715ae2e1e1" containerName="maas-api" containerID="cri-o://b7c464b4cfc971ccefcf31b1e7bd6671de604e3cb5042f1886b1174ca21c3ab5" gracePeriod=30 Jul 06 21:48:46.661279 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:46.661256 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:46.732142 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:46.732102 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:46.854560 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:46.854525 2573 generic.go:356] "Generic (PLEG): container finished" podID="36bdb4ef-945f-40dc-a1af-aa715ae2e1e1" containerID="b7c464b4cfc971ccefcf31b1e7bd6671de604e3cb5042f1886b1174ca21c3ab5" exitCode=0 Jul 06 21:48:46.854719 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:46.854596 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" event={"ID":"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1","Type":"ContainerDied","Data":"b7c464b4cfc971ccefcf31b1e7bd6671de604e3cb5042f1886b1174ca21c3ab5"} Jul 06 21:48:46.876639 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:46.876613 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:47.010355 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:47.010262 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-openshift-service-ca\") pod \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\" (UID: \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\") " Jul 06 21:48:47.010798 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:47.010362 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-maas-api-tls\") pod \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\" (UID: \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\") " Jul 06 21:48:47.010798 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:47.010392 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttcvd\" (UniqueName: \"kubernetes.io/projected/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-kube-api-access-ttcvd\") pod \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\" (UID: \"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1\") " Jul 06 21:48:47.010798 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:47.010634 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "36bdb4ef-945f-40dc-a1af-aa715ae2e1e1" (UID: "36bdb4ef-945f-40dc-a1af-aa715ae2e1e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:48:47.012507 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:47.012483 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-kube-api-access-ttcvd" (OuterVolumeSpecName: "kube-api-access-ttcvd") pod "36bdb4ef-945f-40dc-a1af-aa715ae2e1e1" (UID: "36bdb4ef-945f-40dc-a1af-aa715ae2e1e1"). InnerVolumeSpecName "kube-api-access-ttcvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:48:47.012673 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:47.012649 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "36bdb4ef-945f-40dc-a1af-aa715ae2e1e1" (UID: "36bdb4ef-945f-40dc-a1af-aa715ae2e1e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:48:47.111141 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:47.111105 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-maas-api-tls\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:48:47.111141 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:47.111136 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttcvd\" (UniqueName: \"kubernetes.io/projected/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-kube-api-access-ttcvd\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:48:47.111141 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:47.111147 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1-openshift-service-ca\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:48:47.860309 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:47.860282 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" Jul 06 21:48:47.860458 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:47.860280 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" event={"ID":"36bdb4ef-945f-40dc-a1af-aa715ae2e1e1","Type":"ContainerDied","Data":"715350cf6cfb6b3d20017283757628a63040f842029c1a1ca140d81d1cc69527"} Jul 06 21:48:47.860458 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:47.860408 2573 scope.go:117] "RemoveContainer" containerID="b7c464b4cfc971ccefcf31b1e7bd6671de604e3cb5042f1886b1174ca21c3ab5" Jul 06 21:48:47.881447 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:47.881421 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7"] Jul 06 21:48:47.884480 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:47.884460 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7"] Jul 06 21:48:48.217420 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:48.217342 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36bdb4ef-945f-40dc-a1af-aa715ae2e1e1" path="/var/lib/kubelet/pods/36bdb4ef-945f-40dc-a1af-aa715ae2e1e1/volumes" Jul 06 21:48:49.811779 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:49.811728 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-9f64f2fa-68b5d54fd6-7pdw7" podUID="36bdb4ef-945f-40dc-a1af-aa715ae2e1e1" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.30:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 06 21:48:53.090638 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:53.090601 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:53.177529 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:53.177496 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:53.387024 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:53.386955 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:53.626240 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:53.626205 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:53.765881 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:53.765806 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:53.853304 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:53.853275 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:53.966730 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:53.966697 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:48:59.988237 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:48:59.988202 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:00.097455 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:00.097421 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:00.198933 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:00.198890 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:00.319113 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:00.319078 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:06.045280 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:06.045240 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:06.125355 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:06.125324 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:08.148717 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:08.148683 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:08.249781 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:08.249751 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:12.922881 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:12.922827 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:13.245017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:13.244893 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:13.676761 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:13.676731 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:16.618766 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.618731 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:16.696776 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.696741 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn"] Jul 06 21:49:16.697167 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.697149 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36bdb4ef-945f-40dc-a1af-aa715ae2e1e1" containerName="maas-api" Jul 06 21:49:16.697264 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.697169 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="36bdb4ef-945f-40dc-a1af-aa715ae2e1e1" containerName="maas-api" Jul 06 21:49:16.697314 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.697265 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="36bdb4ef-945f-40dc-a1af-aa715ae2e1e1" containerName="maas-api" Jul 06 21:49:16.700060 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.700038 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:49:16.702773 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.702753 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-f377e18b-serving-cert\"" Jul 06 21:49:16.711483 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.711460 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn"] Jul 06 21:49:16.728996 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.728908 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:16.748138 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.748111 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d7z7\" (UniqueName: \"kubernetes.io/projected/ab445213-1ff1-441e-9280-359f40b0e19e-kube-api-access-2d7z7\") pod \"maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn\" (UID: \"ab445213-1ff1-441e-9280-359f40b0e19e\") " pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:49:16.748229 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.748163 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab445213-1ff1-441e-9280-359f40b0e19e-openshift-service-ca\") pod \"maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn\" (UID: \"ab445213-1ff1-441e-9280-359f40b0e19e\") " pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:49:16.748269 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.748226 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab445213-1ff1-441e-9280-359f40b0e19e-maas-api-tls\") pod \"maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn\" (UID: \"ab445213-1ff1-441e-9280-359f40b0e19e\") " pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:49:16.800709 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.800685 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:16.849179 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.849155 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d7z7\" (UniqueName: \"kubernetes.io/projected/ab445213-1ff1-441e-9280-359f40b0e19e-kube-api-access-2d7z7\") pod \"maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn\" (UID: \"ab445213-1ff1-441e-9280-359f40b0e19e\") " pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:49:16.849303 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.849196 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab445213-1ff1-441e-9280-359f40b0e19e-openshift-service-ca\") pod \"maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn\" (UID: \"ab445213-1ff1-441e-9280-359f40b0e19e\") " pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:49:16.849303 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.849241 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab445213-1ff1-441e-9280-359f40b0e19e-maas-api-tls\") pod \"maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn\" (UID: \"ab445213-1ff1-441e-9280-359f40b0e19e\") " pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:49:16.849434 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:49:16.849417 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-f377e18b-serving-cert: secret "maas-api-e2e-mt-f377e18b-serving-cert" not found Jul 06 21:49:16.849510 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:49:16.849499 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab445213-1ff1-441e-9280-359f40b0e19e-maas-api-tls podName:ab445213-1ff1-441e-9280-359f40b0e19e nodeName:}" failed. No retries permitted until 2026-07-06 21:49:17.34947802 +0000 UTC m=+1665.101101077 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ab445213-1ff1-441e-9280-359f40b0e19e-maas-api-tls") pod "maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" (UID: "ab445213-1ff1-441e-9280-359f40b0e19e") : secret "maas-api-e2e-mt-f377e18b-serving-cert" not found Jul 06 21:49:16.849843 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.849823 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab445213-1ff1-441e-9280-359f40b0e19e-openshift-service-ca\") pod \"maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn\" (UID: \"ab445213-1ff1-441e-9280-359f40b0e19e\") " pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:49:16.859717 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:16.859693 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d7z7\" (UniqueName: \"kubernetes.io/projected/ab445213-1ff1-441e-9280-359f40b0e19e-kube-api-access-2d7z7\") pod \"maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn\" (UID: \"ab445213-1ff1-441e-9280-359f40b0e19e\") " pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:49:17.209268 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:17.209228 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:17.299204 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:17.299171 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:17.354175 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:17.354141 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab445213-1ff1-441e-9280-359f40b0e19e-maas-api-tls\") pod \"maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn\" (UID: \"ab445213-1ff1-441e-9280-359f40b0e19e\") " pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:49:17.356673 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:17.356649 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab445213-1ff1-441e-9280-359f40b0e19e-maas-api-tls\") pod \"maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn\" (UID: \"ab445213-1ff1-441e-9280-359f40b0e19e\") " pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:49:17.397744 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:17.397713 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:17.610289 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:17.610253 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:49:17.746475 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:17.746452 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn"] Jul 06 21:49:17.748970 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:49:17.748922 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab445213_1ff1_441e_9280_359f40b0e19e.slice/crio-1f3502699558b118bdcf6352bb7c812ed2de4fe338b3d492515c9f625f3f5af3 WatchSource:0}: Error finding container 1f3502699558b118bdcf6352bb7c812ed2de4fe338b3d492515c9f625f3f5af3: Status 404 returned error can't find the container with id 1f3502699558b118bdcf6352bb7c812ed2de4fe338b3d492515c9f625f3f5af3 Jul 06 21:49:17.986440 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:17.986366 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" event={"ID":"ab445213-1ff1-441e-9280-359f40b0e19e","Type":"ContainerStarted","Data":"1f3502699558b118bdcf6352bb7c812ed2de4fe338b3d492515c9f625f3f5af3"} Jul 06 21:49:18.992159 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:18.992124 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" event={"ID":"ab445213-1ff1-441e-9280-359f40b0e19e","Type":"ContainerStarted","Data":"4bc9f9f425867a7e21e452a8975a577274210d6f31130be2bd11201b78f3d574"} Jul 06 21:49:18.992598 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:18.992206 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:49:19.009887 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:19.009842 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" podStartSLOduration=2.627810147 podStartE2EDuration="3.009827972s" totalImagesPullingTime="382.017825ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:49:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:49:19.00865377 +0000 UTC m=+1666.760276862" watchObservedRunningTime="2026-07-06 21:49:19.009827972 +0000 UTC m=+1666.761451057" Jul 06 21:49:21.158996 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:21.158940 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:25.001524 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:25.001493 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:49:27.207815 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:27.207774 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:33.137732 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:33.137690 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:33.238602 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:33.238564 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:39.051659 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:39.051625 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:39.150013 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:39.149980 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:59.495523 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:59.495439 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:49:59.779401 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:59.779371 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn"] Jul 06 21:49:59.779658 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:59.779628 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" podUID="ab445213-1ff1-441e-9280-359f40b0e19e" containerName="maas-api" containerID="cri-o://4bc9f9f425867a7e21e452a8975a577274210d6f31130be2bd11201b78f3d574" gracePeriod=30 Jul 06 21:49:59.819737 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:49:59.819711 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:00.018346 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.018320 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:50:00.107196 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.107114 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2d7z7\" (UniqueName: \"kubernetes.io/projected/ab445213-1ff1-441e-9280-359f40b0e19e-kube-api-access-2d7z7\") pod \"ab445213-1ff1-441e-9280-359f40b0e19e\" (UID: \"ab445213-1ff1-441e-9280-359f40b0e19e\") " Jul 06 21:50:00.107334 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.107257 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab445213-1ff1-441e-9280-359f40b0e19e-maas-api-tls\") pod \"ab445213-1ff1-441e-9280-359f40b0e19e\" (UID: \"ab445213-1ff1-441e-9280-359f40b0e19e\") " Jul 06 21:50:00.107334 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.107297 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab445213-1ff1-441e-9280-359f40b0e19e-openshift-service-ca\") pod \"ab445213-1ff1-441e-9280-359f40b0e19e\" (UID: \"ab445213-1ff1-441e-9280-359f40b0e19e\") " Jul 06 21:50:00.107658 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.107632 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab445213-1ff1-441e-9280-359f40b0e19e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ab445213-1ff1-441e-9280-359f40b0e19e" (UID: "ab445213-1ff1-441e-9280-359f40b0e19e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:50:00.109167 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.109139 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab445213-1ff1-441e-9280-359f40b0e19e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ab445213-1ff1-441e-9280-359f40b0e19e" (UID: "ab445213-1ff1-441e-9280-359f40b0e19e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:50:00.109240 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.109201 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab445213-1ff1-441e-9280-359f40b0e19e-kube-api-access-2d7z7" (OuterVolumeSpecName: "kube-api-access-2d7z7") pod "ab445213-1ff1-441e-9280-359f40b0e19e" (UID: "ab445213-1ff1-441e-9280-359f40b0e19e"). InnerVolumeSpecName "kube-api-access-2d7z7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:50:00.177382 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.177356 2573 generic.go:356] "Generic (PLEG): container finished" podID="ab445213-1ff1-441e-9280-359f40b0e19e" containerID="4bc9f9f425867a7e21e452a8975a577274210d6f31130be2bd11201b78f3d574" exitCode=0 Jul 06 21:50:00.177507 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.177397 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" Jul 06 21:50:00.177507 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.177440 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" event={"ID":"ab445213-1ff1-441e-9280-359f40b0e19e","Type":"ContainerDied","Data":"4bc9f9f425867a7e21e452a8975a577274210d6f31130be2bd11201b78f3d574"} Jul 06 21:50:00.177507 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.177500 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" event={"ID":"ab445213-1ff1-441e-9280-359f40b0e19e","Type":"ContainerDied","Data":"1f3502699558b118bdcf6352bb7c812ed2de4fe338b3d492515c9f625f3f5af3"} Jul 06 21:50:00.177618 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.177523 2573 scope.go:117] "RemoveContainer" containerID="4bc9f9f425867a7e21e452a8975a577274210d6f31130be2bd11201b78f3d574" Jul 06 21:50:00.185395 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.185377 2573 scope.go:117] "RemoveContainer" containerID="4bc9f9f425867a7e21e452a8975a577274210d6f31130be2bd11201b78f3d574" Jul 06 21:50:00.185645 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:50:00.185627 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bc9f9f425867a7e21e452a8975a577274210d6f31130be2bd11201b78f3d574\": container with ID starting with 4bc9f9f425867a7e21e452a8975a577274210d6f31130be2bd11201b78f3d574 not found: ID does not exist" containerID="4bc9f9f425867a7e21e452a8975a577274210d6f31130be2bd11201b78f3d574" Jul 06 21:50:00.185710 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.185654 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bc9f9f425867a7e21e452a8975a577274210d6f31130be2bd11201b78f3d574"} err="failed to get container status \"4bc9f9f425867a7e21e452a8975a577274210d6f31130be2bd11201b78f3d574\": rpc error: code = NotFound desc = could not find container \"4bc9f9f425867a7e21e452a8975a577274210d6f31130be2bd11201b78f3d574\": container with ID starting with 4bc9f9f425867a7e21e452a8975a577274210d6f31130be2bd11201b78f3d574 not found: ID does not exist" Jul 06 21:50:00.199918 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.199897 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn"] Jul 06 21:50:00.205444 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.205422 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn"] Jul 06 21:50:00.208210 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.208192 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2d7z7\" (UniqueName: \"kubernetes.io/projected/ab445213-1ff1-441e-9280-359f40b0e19e-kube-api-access-2d7z7\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:50:00.208263 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.208213 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab445213-1ff1-441e-9280-359f40b0e19e-maas-api-tls\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:50:00.208263 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.208222 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab445213-1ff1-441e-9280-359f40b0e19e-openshift-service-ca\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:50:00.217079 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:00.217057 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab445213-1ff1-441e-9280-359f40b0e19e" path="/var/lib/kubelet/pods/ab445213-1ff1-441e-9280-359f40b0e19e/volumes" Jul 06 21:50:02.997433 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:02.997392 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-f377e18b-7d8b84c76f-mqsbn" podUID="ab445213-1ff1-441e-9280-359f40b0e19e" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.31:8443/health\": context deadline exceeded" Jul 06 21:50:05.622917 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:05.622880 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:10.720895 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:10.720865 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:47.845135 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:47.845106 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:48.964278 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:48.964245 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:49.140781 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:49.140746 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:53.790743 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:53.790704 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:53.893226 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:53.893191 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:54.109179 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:54.109104 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:54.346914 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:54.346878 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:55.559401 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:55.559369 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:55.664017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:55.663985 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:55.767142 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:55.767112 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:56.561492 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:56.561458 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:56.656080 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:56.656048 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:56.757479 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:56.757445 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:50:58.191050 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:50:58.191023 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:04.174751 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:04.174719 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:04.238891 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:04.238845 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:04.904292 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:04.904252 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:10.839882 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:10.839847 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:10.946110 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:10.946079 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:11.041785 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:11.041741 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:21.600664 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:21.600626 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:21.938132 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:21.938057 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:34.192307 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:34.192282 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:51:34.194842 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:34.194818 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:51:35.795291 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:35.795256 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:42.960530 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:42.960497 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:49.100920 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:49.100879 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:49.203168 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:49.203134 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:49.499343 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:49.499261 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:49.817154 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:49.817122 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:50.099194 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:50.099104 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:52.598718 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.598682 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:52.667004 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.666972 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg"] Jul 06 21:51:52.667402 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.667384 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab445213-1ff1-441e-9280-359f40b0e19e" containerName="maas-api" Jul 06 21:51:52.667491 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.667405 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab445213-1ff1-441e-9280-359f40b0e19e" containerName="maas-api" Jul 06 21:51:52.667491 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.667483 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab445213-1ff1-441e-9280-359f40b0e19e" containerName="maas-api" Jul 06 21:51:52.670423 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.670403 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:51:52.673575 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.673557 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-ea178e-serving-cert\"" Jul 06 21:51:52.686134 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.686106 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg"] Jul 06 21:51:52.697218 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.697192 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:52.705231 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.705209 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5vbj\" (UniqueName: \"kubernetes.io/projected/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-kube-api-access-t5vbj\") pod \"maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg\" (UID: \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\") " pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:51:52.705335 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.705240 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg\" (UID: \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\") " pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:51:52.705335 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.705322 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-maas-api-tls\") pod \"maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg\" (UID: \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\") " pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:51:52.801364 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.801331 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:52.806483 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.806454 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-maas-api-tls\") pod \"maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg\" (UID: \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\") " pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:51:52.806594 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.806546 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5vbj\" (UniqueName: \"kubernetes.io/projected/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-kube-api-access-t5vbj\") pod \"maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg\" (UID: \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\") " pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:51:52.806594 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.806575 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg\" (UID: \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\") " pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:51:52.806679 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:51:52.806607 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-ea178e-serving-cert: secret "maas-api-e2e-inf-a-ea178e-serving-cert" not found Jul 06 21:51:52.806726 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:51:52.806684 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-maas-api-tls podName:999cbd01-f5ad-4f3c-8d9f-54e398a118f6 nodeName:}" failed. No retries permitted until 2026-07-06 21:51:53.306662565 +0000 UTC m=+1821.058285615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-maas-api-tls") pod "maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" (UID: "999cbd01-f5ad-4f3c-8d9f-54e398a118f6") : secret "maas-api-e2e-inf-a-ea178e-serving-cert" not found Jul 06 21:51:52.807288 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.807271 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg\" (UID: \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\") " pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:51:52.816502 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:52.816484 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5vbj\" (UniqueName: \"kubernetes.io/projected/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-kube-api-access-t5vbj\") pod \"maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg\" (UID: \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\") " pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:51:53.077502 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:53.077469 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:53.262122 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:53.262084 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:53.309991 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:53.309940 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-maas-api-tls\") pod \"maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg\" (UID: \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\") " pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:51:53.312266 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:53.312242 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-maas-api-tls\") pod \"maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg\" (UID: \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\") " pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:51:53.362371 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:53.362299 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:53.580889 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:53.580854 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:51:53.713604 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:53.713579 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg"] Jul 06 21:51:53.716088 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:51:53.716062 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod999cbd01_f5ad_4f3c_8d9f_54e398a118f6.slice/crio-c06c7b3349707b1f4a73a439a8288fb084725b7cd5ecb45f844ceec7f5326f0f WatchSource:0}: Error finding container c06c7b3349707b1f4a73a439a8288fb084725b7cd5ecb45f844ceec7f5326f0f: Status 404 returned error can't find the container with id c06c7b3349707b1f4a73a439a8288fb084725b7cd5ecb45f844ceec7f5326f0f Jul 06 21:51:53.717422 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:53.717402 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 21:51:54.653033 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:54.652989 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" event={"ID":"999cbd01-f5ad-4f3c-8d9f-54e398a118f6","Type":"ContainerStarted","Data":"745f2a518f165b4deffadf7fb8abbf1cb9df33a27cc22cd67779af35b232cde8"} Jul 06 21:51:54.653229 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:54.653046 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" event={"ID":"999cbd01-f5ad-4f3c-8d9f-54e398a118f6","Type":"ContainerStarted","Data":"c06c7b3349707b1f4a73a439a8288fb084725b7cd5ecb45f844ceec7f5326f0f"} Jul 06 21:51:54.653229 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:54.653102 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:51:54.671650 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:54.671601 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" podStartSLOduration=2.253573732 podStartE2EDuration="2.671585731s" totalImagesPullingTime="418.011999ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:51:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:51:54.670480615 +0000 UTC m=+1822.422103696" watchObservedRunningTime="2026-07-06 21:51:54.671585731 +0000 UTC m=+1822.423208796" Jul 06 21:51:57.042484 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:57.042448 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:57.417092 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:57.417009 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:57.788403 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:57.788366 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:51:58.282749 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:51:58.282721 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:52:00.514747 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.514712 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:52:00.583250 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.583218 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8"] Jul 06 21:52:00.586587 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.586565 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:52:00.589223 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.589198 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-f79c56-serving-cert\"" Jul 06 21:52:00.602052 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.602028 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8"] Jul 06 21:52:00.615967 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.615911 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:52:00.662995 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.662936 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:52:00.667585 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.667565 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-std2x\" (UniqueName: \"kubernetes.io/projected/a47984d2-4382-4494-aa54-18240beab7d1-kube-api-access-std2x\") pod \"maas-api-e2e-inf-b-f79c56-75875df55f-xthb8\" (UID: \"a47984d2-4382-4494-aa54-18240beab7d1\") " pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:52:00.667647 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.667618 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47984d2-4382-4494-aa54-18240beab7d1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f79c56-75875df55f-xthb8\" (UID: \"a47984d2-4382-4494-aa54-18240beab7d1\") " pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:52:00.667683 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.667671 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47984d2-4382-4494-aa54-18240beab7d1-maas-api-tls\") pod \"maas-api-e2e-inf-b-f79c56-75875df55f-xthb8\" (UID: \"a47984d2-4382-4494-aa54-18240beab7d1\") " pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:52:00.720256 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.720210 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:52:00.768938 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.768860 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47984d2-4382-4494-aa54-18240beab7d1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f79c56-75875df55f-xthb8\" (UID: \"a47984d2-4382-4494-aa54-18240beab7d1\") " pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:52:00.768938 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.768927 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47984d2-4382-4494-aa54-18240beab7d1-maas-api-tls\") pod \"maas-api-e2e-inf-b-f79c56-75875df55f-xthb8\" (UID: \"a47984d2-4382-4494-aa54-18240beab7d1\") " pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:52:00.769213 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.768993 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-std2x\" (UniqueName: \"kubernetes.io/projected/a47984d2-4382-4494-aa54-18240beab7d1-kube-api-access-std2x\") pod \"maas-api-e2e-inf-b-f79c56-75875df55f-xthb8\" (UID: \"a47984d2-4382-4494-aa54-18240beab7d1\") " pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:52:00.769213 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:52:00.769130 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-f79c56-serving-cert: secret "maas-api-e2e-inf-b-f79c56-serving-cert" not found Jul 06 21:52:00.769213 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:52:00.769199 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a47984d2-4382-4494-aa54-18240beab7d1-maas-api-tls podName:a47984d2-4382-4494-aa54-18240beab7d1 nodeName:}" failed. No retries permitted until 2026-07-06 21:52:01.269178318 +0000 UTC m=+1829.020801361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a47984d2-4382-4494-aa54-18240beab7d1-maas-api-tls") pod "maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" (UID: "a47984d2-4382-4494-aa54-18240beab7d1") : secret "maas-api-e2e-inf-b-f79c56-serving-cert" not found Jul 06 21:52:00.769600 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.769582 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47984d2-4382-4494-aa54-18240beab7d1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f79c56-75875df55f-xthb8\" (UID: \"a47984d2-4382-4494-aa54-18240beab7d1\") " pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:52:00.781332 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:00.781312 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-std2x\" (UniqueName: \"kubernetes.io/projected/a47984d2-4382-4494-aa54-18240beab7d1-kube-api-access-std2x\") pod \"maas-api-e2e-inf-b-f79c56-75875df55f-xthb8\" (UID: \"a47984d2-4382-4494-aa54-18240beab7d1\") " pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:52:01.239758 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:01.239720 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:52:01.274437 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:01.274403 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47984d2-4382-4494-aa54-18240beab7d1-maas-api-tls\") pod \"maas-api-e2e-inf-b-f79c56-75875df55f-xthb8\" (UID: \"a47984d2-4382-4494-aa54-18240beab7d1\") " pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:52:01.276910 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:01.276879 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47984d2-4382-4494-aa54-18240beab7d1-maas-api-tls\") pod \"maas-api-e2e-inf-b-f79c56-75875df55f-xthb8\" (UID: \"a47984d2-4382-4494-aa54-18240beab7d1\") " pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:52:01.339898 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:01.339860 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:52:01.440481 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:01.440445 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:52:01.498911 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:01.498834 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:52:01.637503 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:01.637359 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8"] Jul 06 21:52:01.640321 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:52:01.640296 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda47984d2_4382_4494_aa54_18240beab7d1.slice/crio-9dc8b79a10c9c0bb758ba11e25de5b83cde039c5abbb9289791a95c6d9af7399 WatchSource:0}: Error finding container 9dc8b79a10c9c0bb758ba11e25de5b83cde039c5abbb9289791a95c6d9af7399: Status 404 returned error can't find the container with id 9dc8b79a10c9c0bb758ba11e25de5b83cde039c5abbb9289791a95c6d9af7399 Jul 06 21:52:01.691769 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:01.691729 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" event={"ID":"a47984d2-4382-4494-aa54-18240beab7d1","Type":"ContainerStarted","Data":"9dc8b79a10c9c0bb758ba11e25de5b83cde039c5abbb9289791a95c6d9af7399"} Jul 06 21:52:02.697036 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:02.696996 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" event={"ID":"a47984d2-4382-4494-aa54-18240beab7d1","Type":"ContainerStarted","Data":"dcbfd4a9d9165ea1118663d60abcfcc34fc73ada7cae1d99ded4355929b335e7"} Jul 06 21:52:02.697424 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:02.697061 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:52:02.721236 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:02.721184 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" podStartSLOduration=2.3372968370000002 podStartE2EDuration="2.721169151s" totalImagesPullingTime="383.872314ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:52:02.71902572 +0000 UTC m=+1830.470648784" watchObservedRunningTime="2026-07-06 21:52:02.721169151 +0000 UTC m=+1830.472792215" Jul 06 21:52:05.021215 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:05.021131 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:52:08.707175 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:08.707148 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:52:11.098030 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:11.097975 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:52:14.213814 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:14.213784 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:52:14.324700 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:14.324662 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:52:21.986882 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:21.986818 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:52:22.088240 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:22.088208 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:52:55.055368 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:52:55.055287 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:13.955493 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:13.955454 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:14.903470 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:14.903432 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:15.089267 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:15.089227 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:15.493791 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:15.493759 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:15.609087 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:15.609052 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:15.710932 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:15.710892 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:15.843425 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:15.843395 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg"] Jul 06 21:53:15.843683 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:15.843658 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" podUID="999cbd01-f5ad-4f3c-8d9f-54e398a118f6" containerName="maas-api" containerID="cri-o://745f2a518f165b4deffadf7fb8abbf1cb9df33a27cc22cd67779af35b232cde8" gracePeriod=30 Jul 06 21:53:15.870830 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:15.870810 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:16.006638 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:16.006604 2573 generic.go:356] "Generic (PLEG): container finished" podID="999cbd01-f5ad-4f3c-8d9f-54e398a118f6" containerID="745f2a518f165b4deffadf7fb8abbf1cb9df33a27cc22cd67779af35b232cde8" exitCode=0 Jul 06 21:53:16.006802 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:16.006672 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" event={"ID":"999cbd01-f5ad-4f3c-8d9f-54e398a118f6","Type":"ContainerDied","Data":"745f2a518f165b4deffadf7fb8abbf1cb9df33a27cc22cd67779af35b232cde8"} Jul 06 21:53:16.088609 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:16.088586 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:53:16.187521 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:16.187416 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-maas-api-tls\") pod \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\" (UID: \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\") " Jul 06 21:53:16.187521 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:16.187463 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-openshift-service-ca\") pod \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\" (UID: \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\") " Jul 06 21:53:16.188054 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:16.187528 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5vbj\" (UniqueName: \"kubernetes.io/projected/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-kube-api-access-t5vbj\") pod \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\" (UID: \"999cbd01-f5ad-4f3c-8d9f-54e398a118f6\") " Jul 06 21:53:16.188054 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:16.187835 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "999cbd01-f5ad-4f3c-8d9f-54e398a118f6" (UID: "999cbd01-f5ad-4f3c-8d9f-54e398a118f6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:53:16.189711 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:16.189690 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-kube-api-access-t5vbj" (OuterVolumeSpecName: "kube-api-access-t5vbj") pod "999cbd01-f5ad-4f3c-8d9f-54e398a118f6" (UID: "999cbd01-f5ad-4f3c-8d9f-54e398a118f6"). InnerVolumeSpecName "kube-api-access-t5vbj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:53:16.189792 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:16.189748 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "999cbd01-f5ad-4f3c-8d9f-54e398a118f6" (UID: "999cbd01-f5ad-4f3c-8d9f-54e398a118f6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:53:16.288959 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:16.288913 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5vbj\" (UniqueName: \"kubernetes.io/projected/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-kube-api-access-t5vbj\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:53:16.289138 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:16.288976 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-maas-api-tls\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:53:16.289138 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:16.289000 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/999cbd01-f5ad-4f3c-8d9f-54e398a118f6-openshift-service-ca\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:53:17.012080 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:17.012054 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" Jul 06 21:53:17.012250 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:17.012050 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg" event={"ID":"999cbd01-f5ad-4f3c-8d9f-54e398a118f6","Type":"ContainerDied","Data":"c06c7b3349707b1f4a73a439a8288fb084725b7cd5ecb45f844ceec7f5326f0f"} Jul 06 21:53:17.012250 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:17.012191 2573 scope.go:117] "RemoveContainer" containerID="745f2a518f165b4deffadf7fb8abbf1cb9df33a27cc22cd67779af35b232cde8" Jul 06 21:53:17.034253 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:17.034218 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg"] Jul 06 21:53:17.037288 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:17.037265 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ea178e-64c9795c69-dcxpg"] Jul 06 21:53:18.217492 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:18.217456 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="999cbd01-f5ad-4f3c-8d9f-54e398a118f6" path="/var/lib/kubelet/pods/999cbd01-f5ad-4f3c-8d9f-54e398a118f6/volumes" Jul 06 21:53:20.594961 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:20.594906 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:57.541091 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:57.541055 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:57.847176 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:57.847098 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:58.098797 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:58.098713 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:58.298988 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:58.298954 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:58.791753 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:58.791718 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:58.890740 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:58.890702 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:58.992914 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:58.992878 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:59.105758 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:59.105670 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8"] Jul 06 21:53:59.106217 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:59.106184 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" podUID="a47984d2-4382-4494-aa54-18240beab7d1" containerName="maas-api" containerID="cri-o://dcbfd4a9d9165ea1118663d60abcfcc34fc73ada7cae1d99ded4355929b335e7" gracePeriod=30 Jul 06 21:53:59.136277 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:59.136225 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:53:59.352233 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:59.352205 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:53:59.449718 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:59.449634 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47984d2-4382-4494-aa54-18240beab7d1-openshift-service-ca\") pod \"a47984d2-4382-4494-aa54-18240beab7d1\" (UID: \"a47984d2-4382-4494-aa54-18240beab7d1\") " Jul 06 21:53:59.449718 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:59.449673 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47984d2-4382-4494-aa54-18240beab7d1-maas-api-tls\") pod \"a47984d2-4382-4494-aa54-18240beab7d1\" (UID: \"a47984d2-4382-4494-aa54-18240beab7d1\") " Jul 06 21:53:59.449897 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:59.449746 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-std2x\" (UniqueName: \"kubernetes.io/projected/a47984d2-4382-4494-aa54-18240beab7d1-kube-api-access-std2x\") pod \"a47984d2-4382-4494-aa54-18240beab7d1\" (UID: \"a47984d2-4382-4494-aa54-18240beab7d1\") " Jul 06 21:53:59.450074 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:59.450048 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a47984d2-4382-4494-aa54-18240beab7d1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a47984d2-4382-4494-aa54-18240beab7d1" (UID: "a47984d2-4382-4494-aa54-18240beab7d1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:53:59.451739 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:59.451711 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a47984d2-4382-4494-aa54-18240beab7d1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a47984d2-4382-4494-aa54-18240beab7d1" (UID: "a47984d2-4382-4494-aa54-18240beab7d1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:53:59.451862 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:59.451827 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a47984d2-4382-4494-aa54-18240beab7d1-kube-api-access-std2x" (OuterVolumeSpecName: "kube-api-access-std2x") pod "a47984d2-4382-4494-aa54-18240beab7d1" (UID: "a47984d2-4382-4494-aa54-18240beab7d1"). InnerVolumeSpecName "kube-api-access-std2x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:53:59.550284 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:59.550247 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-std2x\" (UniqueName: \"kubernetes.io/projected/a47984d2-4382-4494-aa54-18240beab7d1-kube-api-access-std2x\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:53:59.550284 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:59.550278 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47984d2-4382-4494-aa54-18240beab7d1-openshift-service-ca\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:53:59.550284 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:53:59.550292 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47984d2-4382-4494-aa54-18240beab7d1-maas-api-tls\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:54:00.190266 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:00.190229 2573 generic.go:356] "Generic (PLEG): container finished" podID="a47984d2-4382-4494-aa54-18240beab7d1" containerID="dcbfd4a9d9165ea1118663d60abcfcc34fc73ada7cae1d99ded4355929b335e7" exitCode=0 Jul 06 21:54:00.190742 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:00.190281 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" event={"ID":"a47984d2-4382-4494-aa54-18240beab7d1","Type":"ContainerDied","Data":"dcbfd4a9d9165ea1118663d60abcfcc34fc73ada7cae1d99ded4355929b335e7"} Jul 06 21:54:00.190742 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:00.190299 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" Jul 06 21:54:00.190742 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:00.190349 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8" event={"ID":"a47984d2-4382-4494-aa54-18240beab7d1","Type":"ContainerDied","Data":"9dc8b79a10c9c0bb758ba11e25de5b83cde039c5abbb9289791a95c6d9af7399"} Jul 06 21:54:00.190742 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:00.190371 2573 scope.go:117] "RemoveContainer" containerID="dcbfd4a9d9165ea1118663d60abcfcc34fc73ada7cae1d99ded4355929b335e7" Jul 06 21:54:00.199039 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:00.199023 2573 scope.go:117] "RemoveContainer" containerID="dcbfd4a9d9165ea1118663d60abcfcc34fc73ada7cae1d99ded4355929b335e7" Jul 06 21:54:00.199279 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:54:00.199262 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcbfd4a9d9165ea1118663d60abcfcc34fc73ada7cae1d99ded4355929b335e7\": container with ID starting with dcbfd4a9d9165ea1118663d60abcfcc34fc73ada7cae1d99ded4355929b335e7 not found: ID does not exist" containerID="dcbfd4a9d9165ea1118663d60abcfcc34fc73ada7cae1d99ded4355929b335e7" Jul 06 21:54:00.199319 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:00.199288 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcbfd4a9d9165ea1118663d60abcfcc34fc73ada7cae1d99ded4355929b335e7"} err="failed to get container status \"dcbfd4a9d9165ea1118663d60abcfcc34fc73ada7cae1d99ded4355929b335e7\": rpc error: code = NotFound desc = could not find container \"dcbfd4a9d9165ea1118663d60abcfcc34fc73ada7cae1d99ded4355929b335e7\": container with ID starting with dcbfd4a9d9165ea1118663d60abcfcc34fc73ada7cae1d99ded4355929b335e7 not found: ID does not exist" Jul 06 21:54:00.228455 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:00.228418 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8"] Jul 06 21:54:00.239439 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:00.239407 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-f79c56-75875df55f-xthb8"] Jul 06 21:54:02.217135 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:02.217102 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a47984d2-4382-4494-aa54-18240beab7d1" path="/var/lib/kubelet/pods/a47984d2-4382-4494-aa54-18240beab7d1/volumes" Jul 06 21:54:04.777449 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:04.777398 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:04.874250 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:04.874221 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:05.071954 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:05.071912 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:05.325261 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:05.325152 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:05.718267 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:05.718168 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:06.166590 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:06.166552 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:06.567147 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:06.567116 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:09.348266 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.348232 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:09.417006 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.416965 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl"] Jul 06 21:54:09.417294 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.417280 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="999cbd01-f5ad-4f3c-8d9f-54e398a118f6" containerName="maas-api" Jul 06 21:54:09.417337 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.417296 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="999cbd01-f5ad-4f3c-8d9f-54e398a118f6" containerName="maas-api" Jul 06 21:54:09.417337 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.417318 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a47984d2-4382-4494-aa54-18240beab7d1" containerName="maas-api" Jul 06 21:54:09.417337 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.417324 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a47984d2-4382-4494-aa54-18240beab7d1" containerName="maas-api" Jul 06 21:54:09.417480 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.417386 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a47984d2-4382-4494-aa54-18240beab7d1" containerName="maas-api" Jul 06 21:54:09.417480 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.417395 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="999cbd01-f5ad-4f3c-8d9f-54e398a118f6" containerName="maas-api" Jul 06 21:54:09.420407 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.420382 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:09.423021 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.422998 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-f1c0e1-serving-cert\"" Jul 06 21:54:09.430884 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.430859 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl"] Jul 06 21:54:09.444293 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.444264 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:09.527312 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.527281 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fb86ad-b105-4a1d-9de3-d4efc44713a8-maas-api-tls\") pod \"maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl\" (UID: \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\") " pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:09.527468 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.527316 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fb86ad-b105-4a1d-9de3-d4efc44713a8-openshift-service-ca\") pod \"maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl\" (UID: \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\") " pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:09.527468 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.527394 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtk8r\" (UniqueName: \"kubernetes.io/projected/92fb86ad-b105-4a1d-9de3-d4efc44713a8-kube-api-access-rtk8r\") pod \"maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl\" (UID: \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\") " pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:09.551062 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.551036 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:09.628345 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.628270 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fb86ad-b105-4a1d-9de3-d4efc44713a8-maas-api-tls\") pod \"maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl\" (UID: \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\") " pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:09.628345 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.628311 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fb86ad-b105-4a1d-9de3-d4efc44713a8-openshift-service-ca\") pod \"maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl\" (UID: \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\") " pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:09.628524 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.628360 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtk8r\" (UniqueName: \"kubernetes.io/projected/92fb86ad-b105-4a1d-9de3-d4efc44713a8-kube-api-access-rtk8r\") pod \"maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl\" (UID: \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\") " pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:09.629066 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.629039 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fb86ad-b105-4a1d-9de3-d4efc44713a8-openshift-service-ca\") pod \"maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl\" (UID: \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\") " pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:09.631085 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.631051 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fb86ad-b105-4a1d-9de3-d4efc44713a8-maas-api-tls\") pod \"maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl\" (UID: \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\") " pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:09.640221 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.640195 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtk8r\" (UniqueName: \"kubernetes.io/projected/92fb86ad-b105-4a1d-9de3-d4efc44713a8-kube-api-access-rtk8r\") pod \"maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl\" (UID: \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\") " pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:09.644435 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.644412 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:09.731414 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.731382 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:09.877036 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:09.877012 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl"] Jul 06 21:54:09.878817 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:54:09.878752 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92fb86ad_b105_4a1d_9de3_d4efc44713a8.slice/crio-ff4c9144bdbf53031ead7fad3438ddc93bc8085c12d88f12af4f2794d5c8ead5 WatchSource:0}: Error finding container ff4c9144bdbf53031ead7fad3438ddc93bc8085c12d88f12af4f2794d5c8ead5: Status 404 returned error can't find the container with id ff4c9144bdbf53031ead7fad3438ddc93bc8085c12d88f12af4f2794d5c8ead5 Jul 06 21:54:10.055169 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:10.055135 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:10.155045 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:10.154966 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:10.235755 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:10.235717 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" event={"ID":"92fb86ad-b105-4a1d-9de3-d4efc44713a8","Type":"ContainerStarted","Data":"ff4c9144bdbf53031ead7fad3438ddc93bc8085c12d88f12af4f2794d5c8ead5"} Jul 06 21:54:11.243007 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:11.242977 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" event={"ID":"92fb86ad-b105-4a1d-9de3-d4efc44713a8","Type":"ContainerStarted","Data":"53f0a05ceb22897bbfc7835a35b2224e6de138a82cf6f8d0296e85f6a80862fe"} Jul 06 21:54:11.243397 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:11.243087 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:11.265368 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:11.265330 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" podStartSLOduration=1.763127374 podStartE2EDuration="2.265318245s" totalImagesPullingTime="502.190871ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:54:11.264280416 +0000 UTC m=+1959.015903480" watchObservedRunningTime="2026-07-06 21:54:11.265318245 +0000 UTC m=+1959.016941309" Jul 06 21:54:13.849693 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:13.849653 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:14.353327 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:14.353297 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:14.699896 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:14.699820 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:15.185870 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:15.185839 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:17.253052 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.253025 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:17.590377 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.590340 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:17.727212 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.727182 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm"] Jul 06 21:54:17.730575 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.730554 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:17.733132 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.733107 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-70ee2a-serving-cert\"" Jul 06 21:54:17.745346 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.745324 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:17.764268 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.764243 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm"] Jul 06 21:54:17.793617 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.793593 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttbf9\" (UniqueName: \"kubernetes.io/projected/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-kube-api-access-ttbf9\") pod \"maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm\" (UID: \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\") " pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:17.793747 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.793648 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-maas-api-tls\") pod \"maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm\" (UID: \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\") " pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:17.793747 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.793670 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-openshift-service-ca\") pod \"maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm\" (UID: \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\") " pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:17.895176 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.895087 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-maas-api-tls\") pod \"maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm\" (UID: \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\") " pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:17.895176 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.895132 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-openshift-service-ca\") pod \"maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm\" (UID: \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\") " pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:17.895373 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.895191 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttbf9\" (UniqueName: \"kubernetes.io/projected/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-kube-api-access-ttbf9\") pod \"maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm\" (UID: \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\") " pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:17.895373 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:54:17.895259 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-70ee2a-serving-cert: secret "maas-api-e2e-api-b-70ee2a-serving-cert" not found Jul 06 21:54:17.895373 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:54:17.895335 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-maas-api-tls podName:b1aa7cf0-3b01-4f7b-9044-60beaee3271f nodeName:}" failed. No retries permitted until 2026-07-06 21:54:18.395317161 +0000 UTC m=+1966.146940203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-maas-api-tls") pod "maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" (UID: "b1aa7cf0-3b01-4f7b-9044-60beaee3271f") : secret "maas-api-e2e-api-b-70ee2a-serving-cert" not found Jul 06 21:54:17.895957 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.895924 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-openshift-service-ca\") pod \"maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm\" (UID: \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\") " pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:17.909111 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.909082 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttbf9\" (UniqueName: \"kubernetes.io/projected/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-kube-api-access-ttbf9\") pod \"maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm\" (UID: \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\") " pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:17.935850 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:17.935818 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:18.306745 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:18.306718 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:18.399803 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:18.399769 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-maas-api-tls\") pod \"maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm\" (UID: \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\") " pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:18.402113 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:18.402086 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-maas-api-tls\") pod \"maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm\" (UID: \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\") " pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:18.408782 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:18.408739 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:18.642725 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:18.642640 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:18.774036 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:18.774012 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm"] Jul 06 21:54:18.776604 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:54:18.776578 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1aa7cf0_3b01_4f7b_9044_60beaee3271f.slice/crio-30361909c5626b5daf867aaf114919027022825cc0653148f3b27a9eec3dffac WatchSource:0}: Error finding container 30361909c5626b5daf867aaf114919027022825cc0653148f3b27a9eec3dffac: Status 404 returned error can't find the container with id 30361909c5626b5daf867aaf114919027022825cc0653148f3b27a9eec3dffac Jul 06 21:54:19.278178 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:19.278147 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" event={"ID":"b1aa7cf0-3b01-4f7b-9044-60beaee3271f","Type":"ContainerStarted","Data":"25e270c3709f83fad2680480a27465198d98e8e1c30abc76d88bd123c18582d9"} Jul 06 21:54:19.278319 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:19.278202 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" event={"ID":"b1aa7cf0-3b01-4f7b-9044-60beaee3271f","Type":"ContainerStarted","Data":"30361909c5626b5daf867aaf114919027022825cc0653148f3b27a9eec3dffac"} Jul 06 21:54:19.278361 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:19.278330 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:19.300039 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:19.299997 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" podStartSLOduration=1.874130699 podStartE2EDuration="2.299981927s" totalImagesPullingTime="425.851228ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:54:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:54:19.299019546 +0000 UTC m=+1967.050642611" watchObservedRunningTime="2026-07-06 21:54:19.299981927 +0000 UTC m=+1967.051604991" Jul 06 21:54:21.841959 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:21.841903 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:25.287036 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:25.286938 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:30.268925 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:30.268883 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:30.729461 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:30.729423 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:30.854744 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:30.854712 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:31.090929 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.090892 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:31.210291 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.210258 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl"] Jul 06 21:54:31.210488 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.210461 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" podUID="92fb86ad-b105-4a1d-9de3-d4efc44713a8" containerName="maas-api" containerID="cri-o://53f0a05ceb22897bbfc7835a35b2224e6de138a82cf6f8d0296e85f6a80862fe" gracePeriod=30 Jul 06 21:54:31.296871 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.296839 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:31.336688 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.336661 2573 generic.go:356] "Generic (PLEG): container finished" podID="92fb86ad-b105-4a1d-9de3-d4efc44713a8" containerID="53f0a05ceb22897bbfc7835a35b2224e6de138a82cf6f8d0296e85f6a80862fe" exitCode=0 Jul 06 21:54:31.336823 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.336733 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" event={"ID":"92fb86ad-b105-4a1d-9de3-d4efc44713a8","Type":"ContainerDied","Data":"53f0a05ceb22897bbfc7835a35b2224e6de138a82cf6f8d0296e85f6a80862fe"} Jul 06 21:54:31.458412 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.458386 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:31.617567 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.617534 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fb86ad-b105-4a1d-9de3-d4efc44713a8-openshift-service-ca\") pod \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\" (UID: \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\") " Jul 06 21:54:31.617764 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.617623 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtk8r\" (UniqueName: \"kubernetes.io/projected/92fb86ad-b105-4a1d-9de3-d4efc44713a8-kube-api-access-rtk8r\") pod \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\" (UID: \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\") " Jul 06 21:54:31.617764 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.617669 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fb86ad-b105-4a1d-9de3-d4efc44713a8-maas-api-tls\") pod \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\" (UID: \"92fb86ad-b105-4a1d-9de3-d4efc44713a8\") " Jul 06 21:54:31.617910 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.617879 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92fb86ad-b105-4a1d-9de3-d4efc44713a8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "92fb86ad-b105-4a1d-9de3-d4efc44713a8" (UID: "92fb86ad-b105-4a1d-9de3-d4efc44713a8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:54:31.619765 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.619740 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92fb86ad-b105-4a1d-9de3-d4efc44713a8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "92fb86ad-b105-4a1d-9de3-d4efc44713a8" (UID: "92fb86ad-b105-4a1d-9de3-d4efc44713a8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:54:31.619881 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.619860 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92fb86ad-b105-4a1d-9de3-d4efc44713a8-kube-api-access-rtk8r" (OuterVolumeSpecName: "kube-api-access-rtk8r") pod "92fb86ad-b105-4a1d-9de3-d4efc44713a8" (UID: "92fb86ad-b105-4a1d-9de3-d4efc44713a8"). InnerVolumeSpecName "kube-api-access-rtk8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:54:31.719067 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.719009 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtk8r\" (UniqueName: \"kubernetes.io/projected/92fb86ad-b105-4a1d-9de3-d4efc44713a8-kube-api-access-rtk8r\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:54:31.719067 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.719030 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fb86ad-b105-4a1d-9de3-d4efc44713a8-maas-api-tls\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:54:31.719067 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:31.719039 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fb86ad-b105-4a1d-9de3-d4efc44713a8-openshift-service-ca\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:54:32.342713 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:32.342685 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" event={"ID":"92fb86ad-b105-4a1d-9de3-d4efc44713a8","Type":"ContainerDied","Data":"ff4c9144bdbf53031ead7fad3438ddc93bc8085c12d88f12af4f2794d5c8ead5"} Jul 06 21:54:32.343150 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:32.342720 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl" Jul 06 21:54:32.343150 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:32.342742 2573 scope.go:117] "RemoveContainer" containerID="53f0a05ceb22897bbfc7835a35b2224e6de138a82cf6f8d0296e85f6a80862fe" Jul 06 21:54:32.360325 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:32.360303 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl"] Jul 06 21:54:32.365646 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:32.365615 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-f1c0e1-5754875b89-vssdl"] Jul 06 21:54:34.008017 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:34.007968 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:34.124711 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:34.124683 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:34.217343 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:34.217315 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92fb86ad-b105-4a1d-9de3-d4efc44713a8" path="/var/lib/kubelet/pods/92fb86ad-b105-4a1d-9de3-d4efc44713a8/volumes" Jul 06 21:54:36.740202 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:36.740167 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:36.809917 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:36.809892 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:37.056291 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:37.056248 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:37.267839 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:37.267801 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:37.458413 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:37.458327 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:37.881329 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:37.881294 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:37.993369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:37.993334 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:38.090245 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.090212 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:38.228173 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.228092 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm"] Jul 06 21:54:38.228310 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.228279 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" podUID="b1aa7cf0-3b01-4f7b-9044-60beaee3271f" containerName="maas-api" containerID="cri-o://25e270c3709f83fad2680480a27465198d98e8e1c30abc76d88bd123c18582d9" gracePeriod=30 Jul 06 21:54:38.258969 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.258913 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:38.371556 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.371529 2573 generic.go:356] "Generic (PLEG): container finished" podID="b1aa7cf0-3b01-4f7b-9044-60beaee3271f" containerID="25e270c3709f83fad2680480a27465198d98e8e1c30abc76d88bd123c18582d9" exitCode=0 Jul 06 21:54:38.371715 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.371606 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" event={"ID":"b1aa7cf0-3b01-4f7b-9044-60beaee3271f","Type":"ContainerDied","Data":"25e270c3709f83fad2680480a27465198d98e8e1c30abc76d88bd123c18582d9"} Jul 06 21:54:38.472168 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.472144 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:38.579935 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.579911 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-openshift-service-ca\") pod \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\" (UID: \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\") " Jul 06 21:54:38.580094 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.580034 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttbf9\" (UniqueName: \"kubernetes.io/projected/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-kube-api-access-ttbf9\") pod \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\" (UID: \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\") " Jul 06 21:54:38.580094 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.580081 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-maas-api-tls\") pod \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\" (UID: \"b1aa7cf0-3b01-4f7b-9044-60beaee3271f\") " Jul 06 21:54:38.580298 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.580274 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b1aa7cf0-3b01-4f7b-9044-60beaee3271f" (UID: "b1aa7cf0-3b01-4f7b-9044-60beaee3271f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 21:54:38.582104 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.582081 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b1aa7cf0-3b01-4f7b-9044-60beaee3271f" (UID: "b1aa7cf0-3b01-4f7b-9044-60beaee3271f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 21:54:38.582199 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.582183 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-kube-api-access-ttbf9" (OuterVolumeSpecName: "kube-api-access-ttbf9") pod "b1aa7cf0-3b01-4f7b-9044-60beaee3271f" (UID: "b1aa7cf0-3b01-4f7b-9044-60beaee3271f"). InnerVolumeSpecName "kube-api-access-ttbf9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 21:54:38.681158 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.681127 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-maas-api-tls\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:54:38.681158 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.681151 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-openshift-service-ca\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:54:38.681158 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:38.681161 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttbf9\" (UniqueName: \"kubernetes.io/projected/b1aa7cf0-3b01-4f7b-9044-60beaee3271f-kube-api-access-ttbf9\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 21:54:39.377622 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:39.377595 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" Jul 06 21:54:39.377622 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:39.377618 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm" event={"ID":"b1aa7cf0-3b01-4f7b-9044-60beaee3271f","Type":"ContainerDied","Data":"30361909c5626b5daf867aaf114919027022825cc0653148f3b27a9eec3dffac"} Jul 06 21:54:39.378170 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:39.377676 2573 scope.go:117] "RemoveContainer" containerID="25e270c3709f83fad2680480a27465198d98e8e1c30abc76d88bd123c18582d9" Jul 06 21:54:39.400143 ip-10-0-131-78 kubenswrapper[2573]: E0706 21:54:39.400120 2573 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1aa7cf0_3b01_4f7b_9044_60beaee3271f.slice\": RecentStats: unable to find data in memory cache]" Jul 06 21:54:39.400656 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:39.400638 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm"] Jul 06 21:54:39.404465 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:39.404444 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-70ee2a-c84bb666d-96kzm"] Jul 06 21:54:40.216878 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:40.216845 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1aa7cf0-3b01-4f7b-9044-60beaee3271f" path="/var/lib/kubelet/pods/b1aa7cf0-3b01-4f7b-9044-60beaee3271f/volumes" Jul 06 21:54:44.374883 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:44.374845 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:44.468283 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:44.468249 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:44.622423 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:44.622373 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:44.829786 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:44.829755 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:45.172606 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:45.172564 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:45.507302 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:45.507218 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:46.497879 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:46.497842 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:48.443035 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.443005 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:48.532921 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.532892 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm"] Jul 06 21:54:48.533264 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.533248 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1aa7cf0-3b01-4f7b-9044-60beaee3271f" containerName="maas-api" Jul 06 21:54:48.533312 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.533268 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1aa7cf0-3b01-4f7b-9044-60beaee3271f" containerName="maas-api" Jul 06 21:54:48.533312 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.533284 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92fb86ad-b105-4a1d-9de3-d4efc44713a8" containerName="maas-api" Jul 06 21:54:48.533312 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.533290 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="92fb86ad-b105-4a1d-9de3-d4efc44713a8" containerName="maas-api" Jul 06 21:54:48.533412 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.533353 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1aa7cf0-3b01-4f7b-9044-60beaee3271f" containerName="maas-api" Jul 06 21:54:48.533412 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.533362 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="92fb86ad-b105-4a1d-9de3-d4efc44713a8" containerName="maas-api" Jul 06 21:54:48.536252 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.536234 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 21:54:48.539862 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.539843 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-ace6c2-serving-cert\"" Jul 06 21:54:48.551978 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.551936 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm"] Jul 06 21:54:48.575450 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.575399 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:48.667868 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.667835 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65757d24-15d5-44f2-bd6e-49ea050669b5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm\" (UID: \"65757d24-15d5-44f2-bd6e-49ea050669b5\") " pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 21:54:48.668041 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.667899 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tf85\" (UniqueName: \"kubernetes.io/projected/65757d24-15d5-44f2-bd6e-49ea050669b5-kube-api-access-4tf85\") pod \"maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm\" (UID: \"65757d24-15d5-44f2-bd6e-49ea050669b5\") " pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 21:54:48.668041 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.667934 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65757d24-15d5-44f2-bd6e-49ea050669b5-maas-api-tls\") pod \"maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm\" (UID: \"65757d24-15d5-44f2-bd6e-49ea050669b5\") " pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 21:54:48.688338 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.688315 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:48.768555 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.768489 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65757d24-15d5-44f2-bd6e-49ea050669b5-maas-api-tls\") pod \"maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm\" (UID: \"65757d24-15d5-44f2-bd6e-49ea050669b5\") " pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 21:54:48.768677 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.768552 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65757d24-15d5-44f2-bd6e-49ea050669b5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm\" (UID: \"65757d24-15d5-44f2-bd6e-49ea050669b5\") " pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 21:54:48.768677 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.768602 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tf85\" (UniqueName: \"kubernetes.io/projected/65757d24-15d5-44f2-bd6e-49ea050669b5-kube-api-access-4tf85\") pod \"maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm\" (UID: \"65757d24-15d5-44f2-bd6e-49ea050669b5\") " pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 21:54:48.769147 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.769127 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65757d24-15d5-44f2-bd6e-49ea050669b5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm\" (UID: \"65757d24-15d5-44f2-bd6e-49ea050669b5\") " pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 21:54:48.770975 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.770958 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65757d24-15d5-44f2-bd6e-49ea050669b5-maas-api-tls\") pod \"maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm\" (UID: \"65757d24-15d5-44f2-bd6e-49ea050669b5\") " pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 21:54:48.778709 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.778688 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tf85\" (UniqueName: \"kubernetes.io/projected/65757d24-15d5-44f2-bd6e-49ea050669b5-kube-api-access-4tf85\") pod \"maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm\" (UID: \"65757d24-15d5-44f2-bd6e-49ea050669b5\") " pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 21:54:48.846577 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.846544 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 21:54:48.979974 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:48.979934 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm"] Jul 06 21:54:48.982144 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:54:48.982120 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65757d24_15d5_44f2_bd6e_49ea050669b5.slice/crio-a34d091afaefeb29108ca48f2375a3c66b41ccdcdae37f7330e20b996768e847 WatchSource:0}: Error finding container a34d091afaefeb29108ca48f2375a3c66b41ccdcdae37f7330e20b996768e847: Status 404 returned error can't find the container with id a34d091afaefeb29108ca48f2375a3c66b41ccdcdae37f7330e20b996768e847 Jul 06 21:54:49.102539 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:49.102501 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:49.210631 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:49.210602 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:49.424876 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:49.424804 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" event={"ID":"65757d24-15d5-44f2-bd6e-49ea050669b5","Type":"ContainerStarted","Data":"a34d091afaefeb29108ca48f2375a3c66b41ccdcdae37f7330e20b996768e847"} Jul 06 21:54:50.431120 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:50.431084 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" event={"ID":"65757d24-15d5-44f2-bd6e-49ea050669b5","Type":"ContainerStarted","Data":"1fddb42cae1a1738eefd49235179abff17c9597d624bb9dff78375598a9d4930"} Jul 06 21:54:50.431486 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:50.431180 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 21:54:50.453534 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:50.453482 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" podStartSLOduration=1.985179558 podStartE2EDuration="2.453465164s" totalImagesPullingTime="468.285606ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:54:50.451725152 +0000 UTC m=+1998.203348215" watchObservedRunningTime="2026-07-06 21:54:50.453465164 +0000 UTC m=+1998.205088228" Jul 06 21:54:52.924938 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:52.924910 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:53.247689 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:53.247615 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:53.572308 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:53.572274 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:53.840085 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:53.839990 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:56.440369 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.440340 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 21:54:56.456506 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.456479 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:56.515872 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.515836 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt"] Jul 06 21:54:56.519962 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.519919 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 21:54:56.524743 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.524723 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-d392e3-serving-cert\"" Jul 06 21:54:56.532121 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.532098 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt"] Jul 06 21:54:56.564666 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.564638 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:56.633794 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.633774 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69eea2e9-4002-4771-b274-83845894746b-maas-api-tls\") pod \"maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt\" (UID: \"69eea2e9-4002-4771-b274-83845894746b\") " pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 21:54:56.633911 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.633813 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqb29\" (UniqueName: \"kubernetes.io/projected/69eea2e9-4002-4771-b274-83845894746b-kube-api-access-nqb29\") pod \"maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt\" (UID: \"69eea2e9-4002-4771-b274-83845894746b\") " pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 21:54:56.633911 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.633870 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69eea2e9-4002-4771-b274-83845894746b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt\" (UID: \"69eea2e9-4002-4771-b274-83845894746b\") " pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 21:54:56.734793 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.734717 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69eea2e9-4002-4771-b274-83845894746b-maas-api-tls\") pod \"maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt\" (UID: \"69eea2e9-4002-4771-b274-83845894746b\") " pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 21:54:56.734793 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.734768 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqb29\" (UniqueName: \"kubernetes.io/projected/69eea2e9-4002-4771-b274-83845894746b-kube-api-access-nqb29\") pod \"maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt\" (UID: \"69eea2e9-4002-4771-b274-83845894746b\") " pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 21:54:56.735010 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.734796 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69eea2e9-4002-4771-b274-83845894746b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt\" (UID: \"69eea2e9-4002-4771-b274-83845894746b\") " pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 21:54:56.735468 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.735450 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69eea2e9-4002-4771-b274-83845894746b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt\" (UID: \"69eea2e9-4002-4771-b274-83845894746b\") " pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 21:54:56.737047 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.737029 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69eea2e9-4002-4771-b274-83845894746b-maas-api-tls\") pod \"maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt\" (UID: \"69eea2e9-4002-4771-b274-83845894746b\") " pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 21:54:56.739495 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.739472 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:56.744919 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.744900 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqb29\" (UniqueName: \"kubernetes.io/projected/69eea2e9-4002-4771-b274-83845894746b-kube-api-access-nqb29\") pod \"maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt\" (UID: \"69eea2e9-4002-4771-b274-83845894746b\") " pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 21:54:56.833764 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.833735 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 21:54:56.908365 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.908332 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:56.964649 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:56.964615 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt"] Jul 06 21:54:56.967554 ip-10-0-131-78 kubenswrapper[2573]: W0706 21:54:56.967524 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69eea2e9_4002_4771_b274_83845894746b.slice/crio-16fb3434041ba661ffd0245033c90a2b78c2fe035b1e10ec4c622a0f764b2904 WatchSource:0}: Error finding container 16fb3434041ba661ffd0245033c90a2b78c2fe035b1e10ec4c622a0f764b2904: Status 404 returned error can't find the container with id 16fb3434041ba661ffd0245033c90a2b78c2fe035b1e10ec4c622a0f764b2904 Jul 06 21:54:57.210226 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:57.210189 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:57.311162 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:57.311135 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:54:57.460659 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:57.460581 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" event={"ID":"69eea2e9-4002-4771-b274-83845894746b","Type":"ContainerStarted","Data":"64ec9c72233711a52a31f17fb80d7c063d8421a58f28281dbe847115065dcb10"} Jul 06 21:54:57.460659 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:57.460640 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" event={"ID":"69eea2e9-4002-4771-b274-83845894746b","Type":"ContainerStarted","Data":"16fb3434041ba661ffd0245033c90a2b78c2fe035b1e10ec4c622a0f764b2904"} Jul 06 21:54:57.461070 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:57.460669 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 21:54:57.486254 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:54:57.486208 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" podStartSLOduration=1.16758804 podStartE2EDuration="1.486192691s" totalImagesPullingTime="318.604651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:54:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:54:57.48461615 +0000 UTC m=+2005.236239216" watchObservedRunningTime="2026-07-06 21:54:57.486192691 +0000 UTC m=+2005.237815755" Jul 06 21:55:00.961348 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:55:00.961311 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:55:03.471455 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:55:03.471427 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 21:55:10.029314 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:55:10.029278 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:55:10.125118 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:55:10.125084 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:55:16.057303 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:55:16.057268 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:55:17.913173 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:55:17.913134 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:55:18.014095 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:55:18.014062 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:55:55.066149 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:55:55.066057 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:56:34.218503 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:56:34.218474 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:56:34.220442 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:56:34.220422 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 21:56:40.808999 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:56:40.808927 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:57:42.017332 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:57:42.017240 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:57:42.122249 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:57:42.122213 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:57:46.901663 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:57:46.901615 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:57:47.827640 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:57:47.827605 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:58:04.540764 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:58:04.540725 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:58:15.307138 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:58:15.307097 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:58:37.833875 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:58:37.833846 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:58:40.098753 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:58:40.098714 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:58:40.156423 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:58:40.156392 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:59:11.782796 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:59:11.782757 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:59:12.830240 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:59:12.830184 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:59:34.736281 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:59:34.736245 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:59:34.837255 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:59:34.837220 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 21:59:36.489771 ip-10-0-131-78 kubenswrapper[2573]: I0706 21:59:36.489736 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:00:00.141173 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:00.141140 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst"] Jul 06 22:00:00.144503 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:00.144484 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst" Jul 06 22:00:00.154662 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:00.154641 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst"] Jul 06 22:00:00.217939 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:00.217909 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvlq2\" (UniqueName: \"kubernetes.io/projected/1682de22-6d64-441b-a049-9a3eafbc980a-kube-api-access-qvlq2\") pod \"maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst\" (UID: \"1682de22-6d64-441b-a049-9a3eafbc980a\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst" Jul 06 22:00:00.318897 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:00.318868 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvlq2\" (UniqueName: \"kubernetes.io/projected/1682de22-6d64-441b-a049-9a3eafbc980a-kube-api-access-qvlq2\") pod \"maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst\" (UID: \"1682de22-6d64-441b-a049-9a3eafbc980a\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst" Jul 06 22:00:00.328608 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:00.328580 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvlq2\" (UniqueName: \"kubernetes.io/projected/1682de22-6d64-441b-a049-9a3eafbc980a-kube-api-access-qvlq2\") pod \"maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst\" (UID: \"1682de22-6d64-441b-a049-9a3eafbc980a\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst" Jul 06 22:00:00.458091 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:00.457991 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst" Jul 06 22:00:00.585235 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:00.585206 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst"] Jul 06 22:00:00.587384 ip-10-0-131-78 kubenswrapper[2573]: W0706 22:00:00.587358 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1682de22_6d64_441b_a049_9a3eafbc980a.slice/crio-c432e6ff27f735b32c140a79358de5289beaf0c7fce0d30e518eebb75b6bba64 WatchSource:0}: Error finding container c432e6ff27f735b32c140a79358de5289beaf0c7fce0d30e518eebb75b6bba64: Status 404 returned error can't find the container with id c432e6ff27f735b32c140a79358de5289beaf0c7fce0d30e518eebb75b6bba64 Jul 06 22:00:00.589289 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:00.589268 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:00:00.731753 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:00.731660 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst" event={"ID":"1682de22-6d64-441b-a049-9a3eafbc980a","Type":"ContainerStarted","Data":"c432e6ff27f735b32c140a79358de5289beaf0c7fce0d30e518eebb75b6bba64"} Jul 06 22:00:02.811512 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:02.811481 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:00:10.925092 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:10.925059 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:00:11.030155 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:11.030118 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:00:17.808032 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:17.807987 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst" event={"ID":"1682de22-6d64-441b-a049-9a3eafbc980a","Type":"ContainerStarted","Data":"172d5ccadf6cdd7485d7f460249870e8c8bb6fff80be752c109501bbf0c936be"} Jul 06 22:00:17.826181 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:17.826125 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst" podStartSLOduration=0.988992002 podStartE2EDuration="17.826108532s" totalImagesPullingTime="16.83711653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:00:17.824101918 +0000 UTC m=+2325.575725009" watchObservedRunningTime="2026-07-06 22:00:17.826108532 +0000 UTC m=+2325.577731596" Jul 06 22:00:18.617646 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:18.617560 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:00:18.726963 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:18.726909 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:00:18.813984 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:18.813928 2573 generic.go:356] "Generic (PLEG): container finished" podID="1682de22-6d64-441b-a049-9a3eafbc980a" containerID="172d5ccadf6cdd7485d7f460249870e8c8bb6fff80be752c109501bbf0c936be" exitCode=0 Jul 06 22:00:18.813984 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:18.813976 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst" event={"ID":"1682de22-6d64-441b-a049-9a3eafbc980a","Type":"ContainerDied","Data":"172d5ccadf6cdd7485d7f460249870e8c8bb6fff80be752c109501bbf0c936be"} Jul 06 22:00:19.944336 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:19.944308 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst" Jul 06 22:00:20.085425 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:20.085335 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvlq2\" (UniqueName: \"kubernetes.io/projected/1682de22-6d64-441b-a049-9a3eafbc980a-kube-api-access-qvlq2\") pod \"1682de22-6d64-441b-a049-9a3eafbc980a\" (UID: \"1682de22-6d64-441b-a049-9a3eafbc980a\") " Jul 06 22:00:20.087430 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:20.087403 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1682de22-6d64-441b-a049-9a3eafbc980a-kube-api-access-qvlq2" (OuterVolumeSpecName: "kube-api-access-qvlq2") pod "1682de22-6d64-441b-a049-9a3eafbc980a" (UID: "1682de22-6d64-441b-a049-9a3eafbc980a"). InnerVolumeSpecName "kube-api-access-qvlq2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:00:20.186882 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:20.186851 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvlq2\" (UniqueName: \"kubernetes.io/projected/1682de22-6d64-441b-a049-9a3eafbc980a-kube-api-access-qvlq2\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 22:00:20.830031 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:20.829999 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst" event={"ID":"1682de22-6d64-441b-a049-9a3eafbc980a","Type":"ContainerDied","Data":"c432e6ff27f735b32c140a79358de5289beaf0c7fce0d30e518eebb75b6bba64"} Jul 06 22:00:20.830031 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:20.830027 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst" Jul 06 22:00:20.830246 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:20.830047 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c432e6ff27f735b32c140a79358de5289beaf0c7fce0d30e518eebb75b6bba64" Jul 06 22:00:42.201794 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:42.201714 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:00:43.207248 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:43.207211 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:00:44.401027 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:44.400975 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:00:45.005776 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.005743 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:00:45.025308 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.025281 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm"] Jul 06 22:00:45.025511 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.025491 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" podUID="65757d24-15d5-44f2-bd6e-49ea050669b5" containerName="maas-api" containerID="cri-o://1fddb42cae1a1738eefd49235179abff17c9597d624bb9dff78375598a9d4930" gracePeriod=30 Jul 06 22:00:45.070322 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.070295 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:00:45.265205 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.265149 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 22:00:45.302220 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.302193 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65757d24-15d5-44f2-bd6e-49ea050669b5-maas-api-tls\") pod \"65757d24-15d5-44f2-bd6e-49ea050669b5\" (UID: \"65757d24-15d5-44f2-bd6e-49ea050669b5\") " Jul 06 22:00:45.302337 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.302231 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tf85\" (UniqueName: \"kubernetes.io/projected/65757d24-15d5-44f2-bd6e-49ea050669b5-kube-api-access-4tf85\") pod \"65757d24-15d5-44f2-bd6e-49ea050669b5\" (UID: \"65757d24-15d5-44f2-bd6e-49ea050669b5\") " Jul 06 22:00:45.302337 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.302282 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65757d24-15d5-44f2-bd6e-49ea050669b5-openshift-service-ca\") pod \"65757d24-15d5-44f2-bd6e-49ea050669b5\" (UID: \"65757d24-15d5-44f2-bd6e-49ea050669b5\") " Jul 06 22:00:45.302760 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.302729 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65757d24-15d5-44f2-bd6e-49ea050669b5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "65757d24-15d5-44f2-bd6e-49ea050669b5" (UID: "65757d24-15d5-44f2-bd6e-49ea050669b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:00:45.304241 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.304210 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65757d24-15d5-44f2-bd6e-49ea050669b5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "65757d24-15d5-44f2-bd6e-49ea050669b5" (UID: "65757d24-15d5-44f2-bd6e-49ea050669b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:00:45.304328 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.304294 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65757d24-15d5-44f2-bd6e-49ea050669b5-kube-api-access-4tf85" (OuterVolumeSpecName: "kube-api-access-4tf85") pod "65757d24-15d5-44f2-bd6e-49ea050669b5" (UID: "65757d24-15d5-44f2-bd6e-49ea050669b5"). InnerVolumeSpecName "kube-api-access-4tf85". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:00:45.402991 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.402934 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65757d24-15d5-44f2-bd6e-49ea050669b5-openshift-service-ca\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 22:00:45.402991 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.402989 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65757d24-15d5-44f2-bd6e-49ea050669b5-maas-api-tls\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 22:00:45.402991 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.402999 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tf85\" (UniqueName: \"kubernetes.io/projected/65757d24-15d5-44f2-bd6e-49ea050669b5-kube-api-access-4tf85\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 22:00:45.950533 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.950497 2573 generic.go:356] "Generic (PLEG): container finished" podID="65757d24-15d5-44f2-bd6e-49ea050669b5" containerID="1fddb42cae1a1738eefd49235179abff17c9597d624bb9dff78375598a9d4930" exitCode=0 Jul 06 22:00:45.950706 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.950544 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" Jul 06 22:00:45.950706 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.950579 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" event={"ID":"65757d24-15d5-44f2-bd6e-49ea050669b5","Type":"ContainerDied","Data":"1fddb42cae1a1738eefd49235179abff17c9597d624bb9dff78375598a9d4930"} Jul 06 22:00:45.950706 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.950631 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm" event={"ID":"65757d24-15d5-44f2-bd6e-49ea050669b5","Type":"ContainerDied","Data":"a34d091afaefeb29108ca48f2375a3c66b41ccdcdae37f7330e20b996768e847"} Jul 06 22:00:45.950706 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.950657 2573 scope.go:117] "RemoveContainer" containerID="1fddb42cae1a1738eefd49235179abff17c9597d624bb9dff78375598a9d4930" Jul 06 22:00:45.959857 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.959842 2573 scope.go:117] "RemoveContainer" containerID="1fddb42cae1a1738eefd49235179abff17c9597d624bb9dff78375598a9d4930" Jul 06 22:00:45.960127 ip-10-0-131-78 kubenswrapper[2573]: E0706 22:00:45.960109 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fddb42cae1a1738eefd49235179abff17c9597d624bb9dff78375598a9d4930\": container with ID starting with 1fddb42cae1a1738eefd49235179abff17c9597d624bb9dff78375598a9d4930 not found: ID does not exist" containerID="1fddb42cae1a1738eefd49235179abff17c9597d624bb9dff78375598a9d4930" Jul 06 22:00:45.960183 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.960136 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fddb42cae1a1738eefd49235179abff17c9597d624bb9dff78375598a9d4930"} err="failed to get container status \"1fddb42cae1a1738eefd49235179abff17c9597d624bb9dff78375598a9d4930\": rpc error: code = NotFound desc = could not find container \"1fddb42cae1a1738eefd49235179abff17c9597d624bb9dff78375598a9d4930\": container with ID starting with 1fddb42cae1a1738eefd49235179abff17c9597d624bb9dff78375598a9d4930 not found: ID does not exist" Jul 06 22:00:45.972460 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.972439 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm"] Jul 06 22:00:45.976702 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:45.976682 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-ace6c2-f6c859c54-2zvzm"] Jul 06 22:00:46.216725 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:46.216643 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65757d24-15d5-44f2-bd6e-49ea050669b5" path="/var/lib/kubelet/pods/65757d24-15d5-44f2-bd6e-49ea050669b5/volumes" Jul 06 22:00:49.848908 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:00:49.848862 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:23.230045 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:23.230010 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:26.729306 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:26.729246 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:26.994294 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:26.994205 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:27.264394 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:27.264309 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:27.420621 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:27.420589 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:27.848263 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:27.848231 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:28.084097 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.084062 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt"] Jul 06 22:01:28.084320 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.084280 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" podUID="69eea2e9-4002-4771-b274-83845894746b" containerName="maas-api" containerID="cri-o://64ec9c72233711a52a31f17fb80d7c063d8421a58f28281dbe847115065dcb10" gracePeriod=30 Jul 06 22:01:28.113625 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.113595 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst"] Jul 06 22:01:28.121348 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.121327 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-d392e3-29722920-4kxst"] Jul 06 22:01:28.161261 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.161229 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:28.218591 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.218553 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1682de22-6d64-441b-a049-9a3eafbc980a" path="/var/lib/kubelet/pods/1682de22-6d64-441b-a049-9a3eafbc980a/volumes" Jul 06 22:01:28.228159 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.228132 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:28.326896 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.326872 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 22:01:28.471463 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.471370 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69eea2e9-4002-4771-b274-83845894746b-openshift-service-ca\") pod \"69eea2e9-4002-4771-b274-83845894746b\" (UID: \"69eea2e9-4002-4771-b274-83845894746b\") " Jul 06 22:01:28.471463 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.471420 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqb29\" (UniqueName: \"kubernetes.io/projected/69eea2e9-4002-4771-b274-83845894746b-kube-api-access-nqb29\") pod \"69eea2e9-4002-4771-b274-83845894746b\" (UID: \"69eea2e9-4002-4771-b274-83845894746b\") " Jul 06 22:01:28.471671 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.471481 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69eea2e9-4002-4771-b274-83845894746b-maas-api-tls\") pod \"69eea2e9-4002-4771-b274-83845894746b\" (UID: \"69eea2e9-4002-4771-b274-83845894746b\") " Jul 06 22:01:28.471766 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.471730 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69eea2e9-4002-4771-b274-83845894746b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "69eea2e9-4002-4771-b274-83845894746b" (UID: "69eea2e9-4002-4771-b274-83845894746b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:01:28.473613 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.473581 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69eea2e9-4002-4771-b274-83845894746b-kube-api-access-nqb29" (OuterVolumeSpecName: "kube-api-access-nqb29") pod "69eea2e9-4002-4771-b274-83845894746b" (UID: "69eea2e9-4002-4771-b274-83845894746b"). InnerVolumeSpecName "kube-api-access-nqb29". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:01:28.473694 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.473676 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69eea2e9-4002-4771-b274-83845894746b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "69eea2e9-4002-4771-b274-83845894746b" (UID: "69eea2e9-4002-4771-b274-83845894746b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:01:28.573072 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.573032 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69eea2e9-4002-4771-b274-83845894746b-maas-api-tls\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 22:01:28.573072 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.573065 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69eea2e9-4002-4771-b274-83845894746b-openshift-service-ca\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 22:01:28.573072 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:28.573079 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqb29\" (UniqueName: \"kubernetes.io/projected/69eea2e9-4002-4771-b274-83845894746b-kube-api-access-nqb29\") on node \"ip-10-0-131-78.ec2.internal\" DevicePath \"\"" Jul 06 22:01:29.145172 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:29.145138 2573 generic.go:356] "Generic (PLEG): container finished" podID="69eea2e9-4002-4771-b274-83845894746b" containerID="64ec9c72233711a52a31f17fb80d7c063d8421a58f28281dbe847115065dcb10" exitCode=0 Jul 06 22:01:29.145709 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:29.145182 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" Jul 06 22:01:29.145709 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:29.145188 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" event={"ID":"69eea2e9-4002-4771-b274-83845894746b","Type":"ContainerDied","Data":"64ec9c72233711a52a31f17fb80d7c063d8421a58f28281dbe847115065dcb10"} Jul 06 22:01:29.145709 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:29.145232 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt" event={"ID":"69eea2e9-4002-4771-b274-83845894746b","Type":"ContainerDied","Data":"16fb3434041ba661ffd0245033c90a2b78c2fe035b1e10ec4c622a0f764b2904"} Jul 06 22:01:29.145709 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:29.145256 2573 scope.go:117] "RemoveContainer" containerID="64ec9c72233711a52a31f17fb80d7c063d8421a58f28281dbe847115065dcb10" Jul 06 22:01:29.154763 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:29.154746 2573 scope.go:117] "RemoveContainer" containerID="64ec9c72233711a52a31f17fb80d7c063d8421a58f28281dbe847115065dcb10" Jul 06 22:01:29.155030 ip-10-0-131-78 kubenswrapper[2573]: E0706 22:01:29.155011 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64ec9c72233711a52a31f17fb80d7c063d8421a58f28281dbe847115065dcb10\": container with ID starting with 64ec9c72233711a52a31f17fb80d7c063d8421a58f28281dbe847115065dcb10 not found: ID does not exist" containerID="64ec9c72233711a52a31f17fb80d7c063d8421a58f28281dbe847115065dcb10" Jul 06 22:01:29.155097 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:29.155044 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64ec9c72233711a52a31f17fb80d7c063d8421a58f28281dbe847115065dcb10"} err="failed to get container status \"64ec9c72233711a52a31f17fb80d7c063d8421a58f28281dbe847115065dcb10\": rpc error: code = NotFound desc = could not find container \"64ec9c72233711a52a31f17fb80d7c063d8421a58f28281dbe847115065dcb10\": container with ID starting with 64ec9c72233711a52a31f17fb80d7c063d8421a58f28281dbe847115065dcb10 not found: ID does not exist" Jul 06 22:01:29.170149 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:29.170126 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt"] Jul 06 22:01:29.174364 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:29.174344 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-d392e3-76f975859f-5vmrt"] Jul 06 22:01:30.222673 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:30.222640 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69eea2e9-4002-4771-b274-83845894746b" path="/var/lib/kubelet/pods/69eea2e9-4002-4771-b274-83845894746b/volumes" Jul 06 22:01:33.889571 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:33.889531 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:34.245044 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:34.245017 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 22:01:34.247717 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:34.247699 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mfjvh_4207b204-e57a-4d2f-b266-87d6831c6ef1/ovn-acl-logging/0.log" Jul 06 22:01:34.277123 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:34.277103 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:34.350539 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:34.350513 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:34.423937 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:34.423865 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:39.811245 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:39.811211 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5cb4b76d8d-m544q_e8108c29-6388-4310-a3a3-03883e97ec70/maas-api/0.log" Jul 06 22:01:40.151436 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:40.151354 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-68768b879b-q4vms_84db4fec-45f0-4192-aedc-c572329b4210/manager/0.log" Jul 06 22:01:40.755735 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:40.755706 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-757974bfb6-hq8xq_a48ed58f-fbd7-48e4-82fd-fd935011c29e/copy-manifests/0.log" Jul 06 22:01:40.762537 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:40.762512 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-757974bfb6-hq8xq_a48ed58f-fbd7-48e4-82fd-fd935011c29e/manager/0.log" Jul 06 22:01:40.875327 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:40.875294 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-hck88_7beb8af2-b284-4876-a6bf-47f7c9488b07/postgres/0.log" Jul 06 22:01:43.474766 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:43.474726 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gz5np_ed271837-a04a-46e4-921f-8acb8f0395f9/discovery/0.log" Jul 06 22:01:44.058074 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:44.058036 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6kkvm_174f6560-4c7f-4f6a-bdde-ee3c86efcf81/payload-processing/0.log" Jul 06 22:01:44.614396 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:44.614370 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-dzwgz_2fbc398e-ba13-4d93-ac4b-a38340c367d4/main/0.log" Jul 06 22:01:44.731677 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:44.731650 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-xqnmb_2ed3c382-62ca-4267-8133-4b5d9971f446/main/0.log" Jul 06 22:01:49.179956 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:49.179905 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:49.261545 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:49.261516 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:01:52.485192 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:52.485163 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zb6q8_ae8aa509-49e8-451b-8987-efcd4d6a4f1f/global-pull-secret-syncer/0.log" Jul 06 22:01:52.529762 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:52.529726 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" Jul 06 22:01:52.646495 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:52.646468 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-78.ec2.internal_02bd88d5d5006b9faa87b2a2edfa6ac4/haproxy/0.log" Jul 06 22:01:55.535714 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:01:55.535612 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dctfx_2692f890-289e-4696-9a23-94c43e3a991c/konnectivity-agent/0.log" limit=1024 Jul 06 22:02:00.242336 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:00.242311 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-pdm59_64ee0da1-b6ec-40eb-880e-0a4b436f6df6/monitoring-plugin/0.log" Jul 06 22:02:00.276278 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:00.276239 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mmphz_7bce37c5-e997-4fa3-b132-e15fc9321a35/node-exporter/0.log" Jul 06 22:02:00.296687 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:00.296661 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mmphz_7bce37c5-e997-4fa3-b132-e15fc9321a35/kube-rbac-proxy/0.log" Jul 06 22:02:00.319466 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:00.319442 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mmphz_7bce37c5-e997-4fa3-b132-e15fc9321a35/init-textfile/0.log" Jul 06 22:02:00.586455 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:00.586424 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8940ca46-00ef-45b6-a98f-8b896d4c12ba/prometheus/0.log" Jul 06 22:02:00.608690 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:00.608665 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8940ca46-00ef-45b6-a98f-8b896d4c12ba/config-reloader/0.log" Jul 06 22:02:00.631216 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:00.631188 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8940ca46-00ef-45b6-a98f-8b896d4c12ba/thanos-sidecar/0.log" Jul 06 22:02:00.654898 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:00.654875 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8940ca46-00ef-45b6-a98f-8b896d4c12ba/kube-rbac-proxy-web/0.log" Jul 06 22:02:00.677420 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:00.677399 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8940ca46-00ef-45b6-a98f-8b896d4c12ba/kube-rbac-proxy/0.log" Jul 06 22:02:00.699692 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:00.699674 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8940ca46-00ef-45b6-a98f-8b896d4c12ba/kube-rbac-proxy-thanos/0.log" Jul 06 22:02:00.719977 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:00.719959 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8940ca46-00ef-45b6-a98f-8b896d4c12ba/init-config-reloader/0.log" Jul 06 22:02:03.024338 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.024308 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r"] Jul 06 22:02:03.024695 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.024614 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65757d24-15d5-44f2-bd6e-49ea050669b5" containerName="maas-api" Jul 06 22:02:03.024695 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.024624 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="65757d24-15d5-44f2-bd6e-49ea050669b5" containerName="maas-api" Jul 06 22:02:03.024695 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.024633 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69eea2e9-4002-4771-b274-83845894746b" containerName="maas-api" Jul 06 22:02:03.024695 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.024638 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="69eea2e9-4002-4771-b274-83845894746b" containerName="maas-api" Jul 06 22:02:03.024695 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.024646 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1682de22-6d64-441b-a049-9a3eafbc980a" containerName="cleanup" Jul 06 22:02:03.024695 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.024652 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1682de22-6d64-441b-a049-9a3eafbc980a" containerName="cleanup" Jul 06 22:02:03.024695 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.024698 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="65757d24-15d5-44f2-bd6e-49ea050669b5" containerName="maas-api" Jul 06 22:02:03.024908 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.024731 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1682de22-6d64-441b-a049-9a3eafbc980a" containerName="cleanup" Jul 06 22:02:03.024908 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.024738 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="69eea2e9-4002-4771-b274-83845894746b" containerName="maas-api" Jul 06 22:02:03.028173 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.028157 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.030837 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.030814 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gx6gm\"/\"openshift-service-ca.crt\"" Jul 06 22:02:03.030837 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.030822 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gx6gm\"/\"default-dockercfg-c9jwm\"" Jul 06 22:02:03.031851 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.031837 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gx6gm\"/\"kube-root-ca.crt\"" Jul 06 22:02:03.040162 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.040141 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r"] Jul 06 22:02:03.159806 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.159780 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60635471-a3c4-4bdb-a704-c36965cc1bf9-sys\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.159993 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.159815 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/60635471-a3c4-4bdb-a704-c36965cc1bf9-proc\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.159993 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.159886 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/60635471-a3c4-4bdb-a704-c36965cc1bf9-podres\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.159993 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.159960 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvnb2\" (UniqueName: \"kubernetes.io/projected/60635471-a3c4-4bdb-a704-c36965cc1bf9-kube-api-access-bvnb2\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.160145 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.160023 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60635471-a3c4-4bdb-a704-c36965cc1bf9-lib-modules\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.260564 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.260528 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60635471-a3c4-4bdb-a704-c36965cc1bf9-sys\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.260720 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.260570 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/60635471-a3c4-4bdb-a704-c36965cc1bf9-proc\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.260720 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.260597 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/60635471-a3c4-4bdb-a704-c36965cc1bf9-podres\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.260720 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.260648 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvnb2\" (UniqueName: \"kubernetes.io/projected/60635471-a3c4-4bdb-a704-c36965cc1bf9-kube-api-access-bvnb2\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.260720 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.260659 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/60635471-a3c4-4bdb-a704-c36965cc1bf9-proc\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.260720 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.260698 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/60635471-a3c4-4bdb-a704-c36965cc1bf9-podres\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.260720 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.260661 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60635471-a3c4-4bdb-a704-c36965cc1bf9-sys\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.260720 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.260704 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60635471-a3c4-4bdb-a704-c36965cc1bf9-lib-modules\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.260991 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.260774 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60635471-a3c4-4bdb-a704-c36965cc1bf9-lib-modules\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.270513 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.270494 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvnb2\" (UniqueName: \"kubernetes.io/projected/60635471-a3c4-4bdb-a704-c36965cc1bf9-kube-api-access-bvnb2\") pod \"perf-node-gather-daemonset-stl8r\" (UID: \"60635471-a3c4-4bdb-a704-c36965cc1bf9\") " pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.337627 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.337602 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:03.467213 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:03.467190 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r"] Jul 06 22:02:03.469185 ip-10-0-131-78 kubenswrapper[2573]: W0706 22:02:03.469155 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod60635471_a3c4_4bdb_a704_c36965cc1bf9.slice/crio-e34647acdee12d2fa46d5c8805aae711f3642ab9538a6d0ee9c4efef59fd447a WatchSource:0}: Error finding container e34647acdee12d2fa46d5c8805aae711f3642ab9538a6d0ee9c4efef59fd447a: Status 404 returned error can't find the container with id e34647acdee12d2fa46d5c8805aae711f3642ab9538a6d0ee9c4efef59fd447a Jul 06 22:02:04.287411 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:04.287375 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" event={"ID":"60635471-a3c4-4bdb-a704-c36965cc1bf9","Type":"ContainerStarted","Data":"394665b46c012272d3469b68b92c816322eee21555077b1649c938909abb9ccb"} Jul 06 22:02:04.287813 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:04.287427 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" event={"ID":"60635471-a3c4-4bdb-a704-c36965cc1bf9","Type":"ContainerStarted","Data":"e34647acdee12d2fa46d5c8805aae711f3642ab9538a6d0ee9c4efef59fd447a"} Jul 06 22:02:04.287813 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:04.287467 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:04.306058 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:04.306012 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" podStartSLOduration=2.305998737 podStartE2EDuration="2.305998737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:02:04.304859378 +0000 UTC m=+2432.056482442" watchObservedRunningTime="2026-07-06 22:02:04.305998737 +0000 UTC m=+2432.057621802" Jul 06 22:02:05.007110 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:05.007078 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qdv89_96f500f5-8245-4d8b-98f4-60c5ff8821ef/dns/0.log" Jul 06 22:02:05.029117 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:05.029095 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qdv89_96f500f5-8245-4d8b-98f4-60c5ff8821ef/kube-rbac-proxy/0.log" Jul 06 22:02:05.120265 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:05.120237 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qmqnl_e7231950-58b8-40d6-ad8d-7a57df972c1f/dns-node-resolver/0.log" Jul 06 22:02:05.709842 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:05.709796 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7djjr_ee182599-bb07-4f59-bc97-1efdff4e0da8/node-ca/0.log" Jul 06 22:02:06.813774 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:06.813742 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gz5np_ed271837-a04a-46e4-921f-8acb8f0395f9/discovery/0.log" Jul 06 22:02:06.990355 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:06.990323 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6kkvm_174f6560-4c7f-4f6a-bdde-ee3c86efcf81/payload-processing/0.log" Jul 06 22:02:07.729223 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:07.729186 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r2mds_594d6a6c-c70c-4642-9c6b-3745fbd64c4f/serve-healthcheck-canary/0.log" Jul 06 22:02:08.280642 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:08.280611 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8wmpm_31c8be7f-93b2-47a2-aaa2-50048aa25e93/kube-rbac-proxy/0.log" Jul 06 22:02:08.301676 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:08.301643 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8wmpm_31c8be7f-93b2-47a2-aaa2-50048aa25e93/exporter/0.log" Jul 06 22:02:08.323915 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:08.323894 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8wmpm_31c8be7f-93b2-47a2-aaa2-50048aa25e93/extractor/0.log" Jul 06 22:02:10.301028 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:10.301001 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gx6gm/perf-node-gather-daemonset-stl8r" Jul 06 22:02:10.666286 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:10.666201 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5cb4b76d8d-m544q_e8108c29-6388-4310-a3a3-03883e97ec70/maas-api/0.log" Jul 06 22:02:10.813116 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:10.813081 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-68768b879b-q4vms_84db4fec-45f0-4192-aedc-c572329b4210/manager/0.log" Jul 06 22:02:11.105435 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:11.105406 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-757974bfb6-hq8xq_a48ed58f-fbd7-48e4-82fd-fd935011c29e/manager/0.log" Jul 06 22:02:11.126207 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:11.126185 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-757974bfb6-hq8xq_a48ed58f-fbd7-48e4-82fd-fd935011c29e/copy-manifests/0.log" Jul 06 22:02:11.152098 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:11.152077 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-hck88_7beb8af2-b284-4876-a6bf-47f7c9488b07/postgres/0.log" Jul 06 22:02:12.684582 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:12.684554 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d694bd4f8-gn5pb_3e904cae-cf94-440e-8181-1305357fe59f/manager/0.log" Jul 06 22:02:13.340019 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:13.339989 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gz5np_ed271837-a04a-46e4-921f-8acb8f0395f9/discovery/0.log" Jul 06 22:02:13.469297 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:13.469222 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6kkvm_174f6560-4c7f-4f6a-bdde-ee3c86efcf81/payload-processing/0.log" Jul 06 22:02:14.732659 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:14.732631 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gz5np_ed271837-a04a-46e4-921f-8acb8f0395f9/discovery/0.log" Jul 06 22:02:14.885100 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:14.885070 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6kkvm_174f6560-4c7f-4f6a-bdde-ee3c86efcf81/payload-processing/0.log" Jul 06 22:02:17.959299 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:17.959268 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-g7mzb_8e23ff36-46e5-4c85-bba2-01ffa8ae2fac/migrator/0.log" Jul 06 22:02:17.978827 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:17.978802 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-g7mzb_8e23ff36-46e5-4c85-bba2-01ffa8ae2fac/graceful-termination/0.log" Jul 06 22:02:19.674182 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:19.674152 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vsbrr_5b843783-33a8-4332-8d96-a21b5abc3a40/kube-multus-additional-cni-plugins/0.log" Jul 06 22:02:19.694586 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:19.694561 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vsbrr_5b843783-33a8-4332-8d96-a21b5abc3a40/egress-router-binary-copy/0.log" Jul 06 22:02:19.714554 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:19.714531 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vsbrr_5b843783-33a8-4332-8d96-a21b5abc3a40/cni-plugins/0.log" Jul 06 22:02:19.735547 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:19.735522 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vsbrr_5b843783-33a8-4332-8d96-a21b5abc3a40/bond-cni-plugin/0.log" Jul 06 22:02:19.755076 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:19.755056 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vsbrr_5b843783-33a8-4332-8d96-a21b5abc3a40/routeoverride-cni/0.log" Jul 06 22:02:19.776465 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:19.776446 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vsbrr_5b843783-33a8-4332-8d96-a21b5abc3a40/whereabouts-cni-bincopy/0.log" Jul 06 22:02:19.799504 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:19.799485 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vsbrr_5b843783-33a8-4332-8d96-a21b5abc3a40/whereabouts-cni/0.log" Jul 06 22:02:19.828888 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:19.828862 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-p2t6l_42a478dd-c1ae-4a3b-898e-9b6386b6bb22/kube-multus/0.log" Jul 06 22:02:19.974375 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:19.974306 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-b8dk6_626f5874-a75d-42b5-b585-a019b92b5283/network-metrics-daemon/0.log" Jul 06 22:02:19.993506 ip-10-0-131-78 kubenswrapper[2573]: I0706 22:02:19.993488 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-b8dk6_626f5874-a75d-42b5-b585-a019b92b5283/kube-rbac-proxy/0.log"