Jul 21 17:58:09.653960 ip-10-0-136-17 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 21 17:58:09.653969 ip-10-0-136-17 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 21 17:58:09.653975 ip-10-0-136-17 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 21 17:58:09.654370 ip-10-0-136-17 systemd[1]: Failed to start Kubernetes Kubelet. Jul 21 17:58:19.850664 ip-10-0-136-17 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 21 17:58:19.850684 ip-10-0-136-17 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot d65adb82862d42c1827bfd5b32e77f64 -- Jul 21 17:59:56.983830 ip-10-0-136-17 systemd[1]: Starting Kubernetes Kubelet... Jul 21 17:59:57.452678 ip-10-0-136-17 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 17:59:57.452678 ip-10-0-136-17 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 21 17:59:57.452678 ip-10-0-136-17 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 17:59:57.452678 ip-10-0-136-17 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 21 17:59:57.452678 ip-10-0-136-17 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 17:59:57.452678 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.346688 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 21 17:59:57.452678 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351661 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 17:59:57.452678 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351678 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 17:59:57.452678 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351681 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 17:59:57.452678 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351685 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 17:59:57.452678 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351688 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 17:59:57.452678 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351691 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 17:59:57.452678 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351694 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 17:59:57.452678 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351697 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 17:59:57.421290 ip-10-0-136-17 systemd[1]: Started Kubernetes Kubelet. Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351700 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351703 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351705 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351708 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351712 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351720 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351724 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351727 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351730 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351733 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351735 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351740 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351744 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351747 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351750 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351753 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351756 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351758 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 17:59:57.713177 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351762 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351766 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351769 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351772 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351774 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351777 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351780 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351784 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351787 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351791 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351794 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351796 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351799 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351803 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351806 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351808 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351811 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351813 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351817 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351820 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 17:59:57.725160 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351822 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351825 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351829 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351831 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351835 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351838 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351840 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351843 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351846 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351849 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351852 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351854 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351857 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351860 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351863 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351865 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351868 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351870 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351873 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351876 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 17:59:57.726056 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351879 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351881 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351884 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351886 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351889 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351891 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351896 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351899 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351902 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351905 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351907 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351910 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351913 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351916 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351918 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351921 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351924 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351926 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351929 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351932 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 17:59:57.726758 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351934 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351937 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351939 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351942 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351945 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351947 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351950 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351953 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351956 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351959 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351962 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351964 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.351967 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352716 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352723 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352726 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352729 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352733 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352736 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 17:59:57.727374 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352740 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352743 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352745 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352748 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352751 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352754 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352757 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352760 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352763 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352765 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352768 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352771 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352774 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352777 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352780 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352783 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352785 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352788 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352791 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352794 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 17:59:57.727996 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352797 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352799 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352802 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352804 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352807 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352809 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352812 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352818 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352822 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352826 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352828 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352831 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352835 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352839 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352843 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352849 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352852 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352855 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352858 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 17:59:57.728616 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352861 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352865 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352868 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352870 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352874 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352877 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352880 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352882 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352885 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352890 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352893 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352896 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352900 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352903 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352905 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352908 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352910 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352913 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352916 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 17:59:57.729226 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352919 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352921 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352923 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352926 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352929 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352931 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352934 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352936 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352939 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352944 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352946 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352949 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352952 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352954 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352957 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352960 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352963 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352965 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352968 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352970 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 17:59:57.729833 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352973 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352975 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352979 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352981 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352984 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352986 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352989 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352991 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352994 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352996 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.352999 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.353001 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.353004 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.353006 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.353009 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354738 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354753 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354756 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354759 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354762 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 17:59:57.730478 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354765 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354768 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354770 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354773 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354776 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354779 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354782 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354784 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354789 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354792 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354795 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354798 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354802 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354804 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354807 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354810 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354812 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354815 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354818 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 21 17:59:57.731100 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354820 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354822 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354825 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354832 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354836 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354839 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354842 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354845 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354854 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354857 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354859 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354862 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354864 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354867 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354870 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354872 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354875 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354878 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354880 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 17:59:57.731732 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354883 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354887 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354890 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354892 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354895 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354897 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354900 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354904 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354908 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354911 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354914 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354917 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354919 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354922 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354924 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354927 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354933 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354935 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354938 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 17:59:57.732333 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354940 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354943 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354945 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354954 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354957 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354959 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354962 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354964 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354967 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354970 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354972 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354975 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354977 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354980 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354984 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354987 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354989 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354992 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354995 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.354997 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 17:59:57.732941 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355000 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355003 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355005 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355008 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355010 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355013 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355015 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355018 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355020 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355025 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355028 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355031 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355033 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355037 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355040 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355042 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355051 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355129 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355137 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355153 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355158 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355162 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 21 17:59:57.733524 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355165 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355170 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355174 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355177 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355182 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355185 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355189 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355192 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355195 2568 flags.go:64] FLAG: --cgroup-root="" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355198 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355201 2568 flags.go:64] FLAG: --client-ca-file="" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355204 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355207 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355214 2568 flags.go:64] FLAG: --cluster-domain="" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355217 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355220 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355223 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355227 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355231 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355235 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355238 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355242 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355245 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355248 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 21 17:59:57.734219 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355251 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355254 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355258 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355270 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355273 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355276 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355279 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355282 2568 flags.go:64] FLAG: --enable-server="true" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355285 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355293 2568 flags.go:64] FLAG: --event-burst="100" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355296 2568 flags.go:64] FLAG: --event-qps="50" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355299 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355303 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355306 2568 flags.go:64] FLAG: --eviction-hard="" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355310 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355313 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355316 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355319 2568 flags.go:64] FLAG: --eviction-soft="" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355322 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355325 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355328 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355331 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355334 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355337 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355340 2568 flags.go:64] FLAG: --feature-gates="" Jul 21 17:59:57.734846 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355343 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355347 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355351 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355354 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355357 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355361 2568 flags.go:64] FLAG: --help="false" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355364 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-136-17.ec2.internal" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355367 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355370 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355373 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355376 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355387 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355390 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355393 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355396 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355399 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355402 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355405 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355408 2568 flags.go:64] FLAG: --kube-reserved="" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355414 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355417 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355420 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355423 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355426 2568 flags.go:64] FLAG: --lock-file="" Jul 21 17:59:57.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355428 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355431 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355434 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355440 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355443 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355446 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355449 2568 flags.go:64] FLAG: --logging-format="text" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355452 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355455 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355458 2568 flags.go:64] FLAG: --manifest-url="" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355462 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355466 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355469 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355473 2568 flags.go:64] FLAG: --max-pods="110" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355476 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355479 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355482 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355485 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355488 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355491 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355516 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355524 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355528 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355531 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 21 17:59:57.736301 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355534 2568 flags.go:64] FLAG: --pod-cidr="" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355537 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355543 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355547 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355550 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355553 2568 flags.go:64] FLAG: --port="10250" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355556 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355559 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06400775776bdb6d8" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355563 2568 flags.go:64] FLAG: --qos-reserved="" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355566 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355569 2568 flags.go:64] FLAG: --register-node="true" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355571 2568 flags.go:64] FLAG: --register-with-taints="" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355575 2568 flags.go:64] FLAG: --registry-burst="10" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355578 2568 flags.go:64] FLAG: --registry-qps="5" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355581 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355584 2568 flags.go:64] FLAG: --reserved-memory="" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355587 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355591 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355594 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355598 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355601 2568 flags.go:64] FLAG: --runonce="false" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355604 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355607 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355610 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355613 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 21 17:59:57.737038 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355615 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355619 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355622 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355625 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355634 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355637 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355640 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355643 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355646 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355649 2568 flags.go:64] FLAG: --system-cgroups="" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355652 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355658 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355661 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355664 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355671 2568 flags.go:64] FLAG: --tls-min-version="" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355674 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355677 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355679 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355682 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355685 2568 flags.go:64] FLAG: --v="2" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355690 2568 flags.go:64] FLAG: --version="false" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355694 2568 flags.go:64] FLAG: --vmodule="" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355697 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.355701 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355862 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 17:59:57.738066 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355867 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355872 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355875 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355878 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355880 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355883 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355886 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355889 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355892 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355895 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355898 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355900 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355910 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355913 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355915 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355918 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355920 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355924 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355927 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355929 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 17:59:57.738778 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355932 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355935 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355937 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355940 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355942 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355945 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355947 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355951 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355954 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355957 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355960 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355962 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355965 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355970 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355973 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355990 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355994 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.355997 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356000 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 17:59:57.941628 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356003 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356005 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356008 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356011 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356014 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356017 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356025 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356028 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356031 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356040 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356044 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356046 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356049 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356052 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356054 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356056 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356059 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356062 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356064 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 17:59:57.942302 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356068 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356072 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356074 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356077 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356079 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356082 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356085 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356089 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356092 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356094 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356097 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356099 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356103 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356106 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356109 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356112 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356116 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356118 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356121 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 17:59:57.942835 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356124 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356132 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356135 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356138 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356142 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356145 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356147 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356150 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356152 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356155 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356158 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356160 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356163 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356165 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356168 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356170 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356173 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356175 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356178 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356180 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 17:59:57.943407 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.356184 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.356752 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.365892 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.365914 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.365935 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.365942 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366039 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366044 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366050 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366056 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366059 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366062 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366065 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366068 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366071 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 17:59:57.943953 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366074 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366077 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366080 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366082 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366085 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366088 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366096 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366099 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366102 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366105 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366107 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366110 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366113 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366116 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366121 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366125 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366128 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366131 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366137 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 17:59:57.944403 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366140 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366143 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366147 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366150 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366152 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366155 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366159 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366163 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366166 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366168 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366172 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366177 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366180 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366183 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366185 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366188 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366191 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366193 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366196 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366200 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 17:59:57.944922 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366203 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366205 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366208 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366211 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366216 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366218 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366221 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366224 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366226 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366229 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366232 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366234 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366237 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366240 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366243 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366246 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366249 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366254 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366258 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366261 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 17:59:57.945490 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366263 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366266 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366269 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366272 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366275 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366277 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366280 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366282 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366285 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366288 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366293 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366297 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366300 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366303 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366307 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366310 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366313 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366316 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366320 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366322 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 17:59:57.946028 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366325 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366328 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366334 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366336 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366339 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366342 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366345 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366350 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366354 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366356 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366360 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.366366 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366596 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366615 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366621 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 17:59:57.946599 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366626 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366630 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366635 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366639 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366644 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366649 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366654 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366658 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366665 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366669 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366674 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366679 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366687 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366701 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366705 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366709 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366717 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366866 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366886 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366891 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 17:59:57.947020 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366895 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366898 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366902 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366906 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366911 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366915 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366919 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366922 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366925 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366928 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366931 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366934 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366937 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366940 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366942 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366945 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366948 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366950 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366953 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366956 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 17:59:57.947594 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366959 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366961 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366964 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366967 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366970 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366972 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366975 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366978 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366980 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366984 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366987 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366990 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366992 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366995 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.366998 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367001 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367003 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367007 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367010 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367012 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 17:59:57.948123 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367015 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367020 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367024 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367026 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367029 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367032 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367035 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367037 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367040 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367043 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367045 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367048 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367051 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367053 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367056 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367059 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367062 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367064 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367067 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367070 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 17:59:57.948686 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367072 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367075 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367078 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367080 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367083 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367085 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367088 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367090 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367093 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367096 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367098 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367101 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367103 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367108 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367111 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 17:59:57.949217 ip-10-0-136-17 kubenswrapper[2568]: W0721 17:59:57.367115 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 17:59:57.949718 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.367121 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 17:59:57.949718 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.367271 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 21 17:59:57.949718 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.370315 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 21 17:59:57.949718 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.371867 2568 server.go:1019] "Starting client certificate rotation" Jul 21 17:59:57.949718 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.371979 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 17:59:57.949718 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.372024 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 17:59:57.949718 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.374263 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 17:59:57.949718 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.375497 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 17:59:57.949718 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.383687 2568 log.go:25] "Validated CRI v1 runtime API" Jul 21 17:59:57.949718 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.390070 2568 log.go:25] "Validated CRI v1 image API" Jul 21 17:59:57.949718 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.391028 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 21 17:59:57.949718 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.394626 2568 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b30f1ea5-c51d-4aa2-969b-f296413790ae:/dev/nvme0n1p4 dde85cb4-22eb-4a36-82fa-869655152176:/dev/nvme0n1p3] Jul 21 17:59:57.950058 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.394643 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 21 17:59:57.950162 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.399207 2568 manager.go:216] Machine: {Timestamp:2026-07-21 17:59:57.398714847 +0000 UTC m=+0.302459746 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101920 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a559d82d9f18ec3009b5f0ac8a74e SystemUUID:ec2a559d-82d9-f18e-c300-9b5f0ac8a74e BootID:d65adb82-862d-42c1-827b-fd5b32e77f64 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:50:c0:d7:80:79 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:50:c0:d7:80:79 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:11:61:7b:dd:ce Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 21 17:59:57.950162 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.399310 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 21 17:59:57.950162 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.399396 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 21 17:59:57.950162 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.401624 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 21 17:59:57.950162 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.401660 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-17.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 21 17:59:57.950162 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.402393 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 21 17:59:57.950162 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.402408 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 21 17:59:57.950162 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.402419 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.402637 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.403063 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.404519 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.408513 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.408655 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.408704 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.408719 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.408737 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.408746 2568 kubelet.go:398] "Adding apiserver pod source" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.408756 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.409193 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.409217 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.413255 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.414737 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.414764 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.415479 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.415492 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.415510 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.415517 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.415526 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.415532 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.415540 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.415546 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.415556 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.415564 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.415572 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.415578 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.416899 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.416906 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.420250 2568 server.go:1268] "Started kubelet" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.420344 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.420418 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.421076 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.422166 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.422489 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-17.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.422647 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.422730 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.423175 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 21 17:59:57.950429 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.426441 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.427436 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-17.ec2.internal.18c45fe6d837f1bc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-17.ec2.internal,UID:ip-10-0-136-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-17.ec2.internal,},FirstTimestamp:2026-07-21 17:59:57.42022086 +0000 UTC m=+0.323965759,LastTimestamp:2026-07-21 17:59:57.42022086 +0000 UTC m=+0.323965759,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-17.ec2.internal,}" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.429486 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.429558 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.429590 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.430126 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.430158 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.430176 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.430305 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.430314 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.430344 2568 factory.go:55] Registering systemd factory Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.430395 2568 factory.go:223] Registration of the systemd container factory successfully Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.430665 2568 factory.go:153] Registering CRI-O factory Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.430677 2568 factory.go:223] Registration of the crio container factory successfully Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.430719 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.430744 2568 factory.go:103] Registering Raw factory Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.430755 2568 manager.go:1196] Started watching for new ooms in manager Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.431187 2568 manager.go:349] Starting recovery of all containers Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.434305 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.437047 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.438266 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.447060 2568 manager.go:354] Recovery completed Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.453249 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.454586 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jj84g" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.455183 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientMemory" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.455211 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.455222 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientPID" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.455793 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.455803 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.455819 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.455804 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.456469 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientMemory" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.456491 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 17:59:57.951461 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.456581 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientPID" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.457375 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-17.ec2.internal.18c45fe6da4da58f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-17.ec2.internal,UID:ip-10-0-136-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-17.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-17.ec2.internal,},FirstTimestamp:2026-07-21 17:59:57.455197583 +0000 UTC m=+0.358942484,LastTimestamp:2026-07-21 17:59:57.455197583 +0000 UTC m=+0.358942484,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-17.ec2.internal,}" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.457905 2568 policy_none.go:49] "None policy: Start" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.457917 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.457928 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.459861 2568 policy_none.go:47] "Start" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.461988 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jj84g" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.468225 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-17.ec2.internal.18c45fe6da4dedf2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-17.ec2.internal,UID:ip-10-0-136-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-17.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-17.ec2.internal,},FirstTimestamp:2026-07-21 17:59:57.455216114 +0000 UTC m=+0.358961013,LastTimestamp:2026-07-21 17:59:57.455216114 +0000 UTC m=+0.358961013,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-17.ec2.internal,}" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.530701 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.562063 2568 manager.go:337] "Starting Device Plugin manager" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.562116 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.562126 2568 server.go:87] "Starting device plugin registration server" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.562387 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.562398 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.562519 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.562631 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.562640 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.563243 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.563292 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.576772 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.576814 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.576835 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.576873 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.579460 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.645147 2568 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-17.ec2.internal\" not found" node="ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.663240 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.664359 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientMemory" Jul 21 17:59:58.117864 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.664386 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.664405 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientPID" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.664434 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.673712 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.673739 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-17.ec2.internal\": node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.676916 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-17.ec2.internal"] Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.676982 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.677957 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientMemory" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.677983 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.677992 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientPID" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.690143 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.690239 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.690270 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.691082 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientMemory" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.691106 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.691120 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientPID" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.691083 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientMemory" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.691195 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.691210 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientPID" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.693352 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.693374 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.693987 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientMemory" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.694015 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.694028 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientPID" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.705750 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.732330 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac6d68d762c0ec53c9f113093b050797-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal\" (UID: \"ac6d68d762c0ec53c9f113093b050797\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.732358 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6d68d762c0ec53c9f113093b050797-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal\" (UID: \"ac6d68d762c0ec53c9f113093b050797\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.119424 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.732390 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0a6fe56b86f91ef9f0ca8f3090eba9bd-config\") pod \"kube-apiserver-proxy-ip-10-0-136-17.ec2.internal\" (UID: \"0a6fe56b86f91ef9f0ca8f3090eba9bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.131086 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.749574 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-17.ec2.internal\" not found" node="ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.131086 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.753778 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-17.ec2.internal\" not found" node="ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.131086 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.806440 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 17:59:58.131086 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.833463 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac6d68d762c0ec53c9f113093b050797-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal\" (UID: \"ac6d68d762c0ec53c9f113093b050797\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.131086 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.833494 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6d68d762c0ec53c9f113093b050797-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal\" (UID: \"ac6d68d762c0ec53c9f113093b050797\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.131086 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.833548 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6d68d762c0ec53c9f113093b050797-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal\" (UID: \"ac6d68d762c0ec53c9f113093b050797\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.131086 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.833550 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac6d68d762c0ec53c9f113093b050797-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal\" (UID: \"ac6d68d762c0ec53c9f113093b050797\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.131086 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.833579 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0a6fe56b86f91ef9f0ca8f3090eba9bd-config\") pod \"kube-apiserver-proxy-ip-10-0-136-17.ec2.internal\" (UID: \"0a6fe56b86f91ef9f0ca8f3090eba9bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.131086 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:57.833600 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0a6fe56b86f91ef9f0ca8f3090eba9bd-config\") pod \"kube-apiserver-proxy-ip-10-0-136-17.ec2.internal\" (UID: \"0a6fe56b86f91ef9f0ca8f3090eba9bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.131086 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:57.906722 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 17:59:58.131086 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:58.007424 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 17:59:58.131086 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:58.052213 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.131086 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:58.057696 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-17.ec2.internal" Jul 21 17:59:58.131086 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:58.108442 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 17:59:59.872034 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:59.844404 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 17:59:59.996091 ip-10-0-136-17 kubenswrapper[2568]: I0721 17:59:59.943457 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 17:59:59.996091 ip-10-0-136-17 kubenswrapper[2568]: E0721 17:59:59.944750 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 18:00:00.008868 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.006521 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 21 18:00:00.008868 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.006724 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 18:00:00.008868 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.006731 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 18:00:00.121531 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:00.044811 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 18:00:00.121531 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.103055 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 17:54:59 +0000 UTC" deadline="2028-01-10 13:17:40.647613688 +0000 UTC" Jul 21 18:00:00.121531 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.103090 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12907h17m40.544525997s" Jul 21 18:00:00.247524 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:00.145204 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 18:00:00.247524 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.188728 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 18:00:00.247524 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.189808 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientMemory" Jul 21 18:00:00.247524 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.189840 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 18:00:00.247524 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.189851 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeHasSufficientPID" Jul 21 18:00:00.247524 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:00.246053 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 18:00:00.357730 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:00.346532 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 18:00:00.471664 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:00.446970 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 18:00:00.589521 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:00.547751 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-17.ec2.internal\" not found" Jul 21 18:00:00.589521 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.549147 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 18:00:00.712945 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.614129 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 18:00:00.712945 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.665542 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-17.ec2.internal" Jul 21 18:00:00.712945 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.684181 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 18:00:00.712945 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.685089 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal" Jul 21 18:00:00.712945 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:00.702372 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.045698 2568 apiserver.go:52] "Watching apiserver" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.052217 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.052542 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-hrk92","kube-system/konnectivity-agent-gtv75","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb","openshift-dns/node-resolver-vlkxx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal","openshift-multus/network-metrics-daemon-kr8ht","openshift-ovn-kubernetes/ovnkube-node-7h7m4","kube-system/kube-apiserver-proxy-ip-10-0-136-17.ec2.internal","openshift-cluster-node-tuning-operator/tuned-fzz5k","openshift-image-registry/node-ca-44h8t","openshift-multus/multus-additional-cni-plugins-q8fqf","openshift-multus/multus-bnghg","openshift-network-diagnostics/network-check-target-4582q","openshift-network-operator/iptables-alerter-zr6bx"] Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.055130 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.057575 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.057591 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.057617 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.057821 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q896n\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.057982 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.058590 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.058602 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.058592 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.060177 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.060510 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.060621 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sh8dg\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.060984 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.061288 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.061317 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.061611 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.062388 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.062485 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.062582 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vlkxx" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.062610 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bqkb7\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.062680 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.064926 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zr6bx" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.066032 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.067581 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:01.088444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.067656 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.067688 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.067878 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.068007 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7dnrj\"" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.068992 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.069169 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.069691 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bscjt\"" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.069915 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.069922 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.072148 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.072154 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tkz44\"" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.072198 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.072312 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.072501 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.074457 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gtv75" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.076631 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-44h8t" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077622 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-etc-openvswitch\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077643 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-node-log\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077660 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-slash\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077675 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/91e663b2-9fae-459f-bbe2-bcd2a022727a-iptables-alerter-script\") pod \"iptables-alerter-zr6bx\" (UID: \"91e663b2-9fae-459f-bbe2-bcd2a022727a\") " pod="openshift-network-operator/iptables-alerter-zr6bx" Jul 21 18:00:01.092533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077695 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.093314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077748 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-socket-dir\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.093314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077765 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-sys-fs\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.093314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077779 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc216396-5088-4fd3-9409-f525712b3b92-hosts-file\") pod \"node-resolver-vlkxx\" (UID: \"dc216396-5088-4fd3-9409-f525712b3b92\") " pod="openshift-dns/node-resolver-vlkxx" Jul 21 18:00:01.093314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077800 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg47k\" (UniqueName: \"kubernetes.io/projected/dc216396-5088-4fd3-9409-f525712b3b92-kube-api-access-cg47k\") pod \"node-resolver-vlkxx\" (UID: \"dc216396-5088-4fd3-9409-f525712b3b92\") " pod="openshift-dns/node-resolver-vlkxx" Jul 21 18:00:01.093314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077816 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph6pd\" (UniqueName: \"kubernetes.io/projected/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-kube-api-access-ph6pd\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.093314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077833 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-registration-dir\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.093314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077848 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-device-dir\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.093314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077870 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-cni-bin\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.093314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077906 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs\") pod \"network-metrics-daemon-kr8ht\" (UID: \"89ffba65-3615-478c-856d-a75629de4606\") " pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:01.093314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077956 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-var-lib-openvswitch\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.093314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.077986 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6746d28c-eb2a-40ec-ab6d-da2e086116dc-env-overrides\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.093314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078010 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.093314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078032 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hsjq\" (UniqueName: \"kubernetes.io/projected/f740821c-eba5-4c6c-bf39-f1ab91c70256-kube-api-access-8hsjq\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.096579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078053 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-systemd-units\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.096579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078075 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-log-socket\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.096579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078095 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-cni-netd\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.096579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078118 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.096579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078142 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-run-openvswitch\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.096579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078191 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-etc-selinux\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.096579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078209 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-kubelet\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.096579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078231 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.096579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078249 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.096579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078264 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-system-cni-dir\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.096579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078278 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-os-release\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.096579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078293 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.096579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078310 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmpxp\" (UniqueName: \"kubernetes.io/projected/91e663b2-9fae-459f-bbe2-bcd2a022727a-kube-api-access-gmpxp\") pod \"iptables-alerter-zr6bx\" (UID: \"91e663b2-9fae-459f-bbe2-bcd2a022727a\") " pod="openshift-network-operator/iptables-alerter-zr6bx" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078325 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmhpt\" (UniqueName: \"kubernetes.io/projected/89ffba65-3615-478c-856d-a75629de4606-kube-api-access-vmhpt\") pod \"network-metrics-daemon-kr8ht\" (UID: \"89ffba65-3615-478c-856d-a75629de4606\") " pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078340 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-run-netns\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078355 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-run-systemd\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078369 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6746d28c-eb2a-40ec-ab6d-da2e086116dc-ovnkube-config\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078383 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6746d28c-eb2a-40ec-ab6d-da2e086116dc-ovnkube-script-lib\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078412 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-cnibin\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078438 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc216396-5088-4fd3-9409-f525712b3b92-tmp-dir\") pod \"node-resolver-vlkxx\" (UID: \"dc216396-5088-4fd3-9409-f525712b3b92\") " pod="openshift-dns/node-resolver-vlkxx" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078455 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/91e663b2-9fae-459f-bbe2-bcd2a022727a-host-slash\") pod \"iptables-alerter-zr6bx\" (UID: \"91e663b2-9fae-459f-bbe2-bcd2a022727a\") " pod="openshift-network-operator/iptables-alerter-zr6bx" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078470 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-run-ovn\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078484 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6746d28c-eb2a-40ec-ab6d-da2e086116dc-ovn-node-metrics-cert\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078499 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5xzl\" (UniqueName: \"kubernetes.io/projected/6746d28c-eb2a-40ec-ab6d-da2e086116dc-kube-api-access-b5xzl\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078513 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-cni-binary-copy\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.078796 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.078851 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:01.097378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.079390 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 21 18:00:01.100594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.079391 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 21 18:00:01.100594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.079391 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 21 18:00:01.100594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.079669 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 21 18:00:01.100594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.079709 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 21 18:00:01.100594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.079770 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8n87w\"" Jul 21 18:00:01.100594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.079817 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zmp9x\"" Jul 21 18:00:01.100594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.081090 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.100594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.083270 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 21 18:00:01.100594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.084068 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 18:00:01.100594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.084286 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-r4mhs\"" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.115237 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lbrvq" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.129805 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lbrvq" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.166117 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.178878 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.178919 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.178961 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-etc-kubernetes\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.178984 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/163da31b-27c0-4e33-b898-4a2066ccea13-serviceca\") pod \"node-ca-44h8t\" (UID: \"163da31b-27c0-4e33-b898-4a2066ccea13\") " pod="openshift-image-registry/node-ca-44h8t" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.178987 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179009 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmhpt\" (UniqueName: \"kubernetes.io/projected/89ffba65-3615-478c-856d-a75629de4606-kube-api-access-vmhpt\") pod \"network-metrics-daemon-kr8ht\" (UID: \"89ffba65-3615-478c-856d-a75629de4606\") " pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179035 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-run-netns\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179109 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6746d28c-eb2a-40ec-ab6d-da2e086116dc-ovnkube-script-lib\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179129 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-run-netns\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179164 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-cnibin\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179263 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-system-cni-dir\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179271 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-cnibin\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.216589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179304 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-systemd\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.221285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179327 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-cni-binary-copy\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.221285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179380 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-run-multus-certs\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.221285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179414 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-sys\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.221285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179438 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7546\" (UniqueName: \"kubernetes.io/projected/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-kube-api-access-m7546\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.221285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179474 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-etc-openvswitch\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.221285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179497 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-node-log\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.221285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179518 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-etc-openvswitch\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.221285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179520 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-dbus\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:01.221285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179551 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-cnibin\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.221285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179571 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-slash\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.221285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179539 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-node-log\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.221285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179587 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/91e663b2-9fae-459f-bbe2-bcd2a022727a-iptables-alerter-script\") pod \"iptables-alerter-zr6bx\" (UID: \"91e663b2-9fae-459f-bbe2-bcd2a022727a\") " pod="openshift-network-operator/iptables-alerter-zr6bx" Jul 21 18:00:01.221285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179620 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.221285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179627 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-slash\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179646 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-socket-dir\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179669 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc216396-5088-4fd3-9409-f525712b3b92-hosts-file\") pod \"node-resolver-vlkxx\" (UID: \"dc216396-5088-4fd3-9409-f525712b3b92\") " pod="openshift-dns/node-resolver-vlkxx" Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179690 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-modprobe-d\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179661 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179707 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-registration-dir\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179751 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-registration-dir\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179766 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-device-dir\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179785 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-hostroot\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179798 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-device-dir\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179805 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs\") pod \"network-metrics-daemon-kr8ht\" (UID: \"89ffba65-3615-478c-856d-a75629de4606\") " pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179769 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dc216396-5088-4fd3-9409-f525712b3b92-hosts-file\") pod \"node-resolver-vlkxx\" (UID: \"dc216396-5088-4fd3-9409-f525712b3b92\") " pod="openshift-dns/node-resolver-vlkxx" Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179821 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-var-lib-openvswitch\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179849 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-var-lib-openvswitch\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.179903 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:01.222115 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179950 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6746d28c-eb2a-40ec-ab6d-da2e086116dc-env-overrides\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.227005 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.179979 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs podName:89ffba65-3615-478c-856d-a75629de4606 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:01.679961466 +0000 UTC m=+2.948272887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs") pod "network-metrics-daemon-kr8ht" (UID: "89ffba65-3615-478c-856d-a75629de4606") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:01.227005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.179994 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-var-lib-cni-multus\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.227005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.180038 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-sysctl-d\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.227005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.180053 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-sysctl-conf\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.227005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.180067 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k9rg\" (UniqueName: \"kubernetes.io/projected/163da31b-27c0-4e33-b898-4a2066ccea13-kube-api-access-2k9rg\") pod \"node-ca-44h8t\" (UID: \"163da31b-27c0-4e33-b898-4a2066ccea13\") " pod="openshift-image-registry/node-ca-44h8t" Jul 21 18:00:01.227005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.180092 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hsjq\" (UniqueName: \"kubernetes.io/projected/f740821c-eba5-4c6c-bf39-f1ab91c70256-kube-api-access-8hsjq\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.227005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.180243 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-cni-netd\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.227005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.180265 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-socket-dir\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.227005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.180118 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-cni-netd\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.227005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.180801 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.227005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.180840 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-os-release\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.227005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.180867 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-multus-conf-dir\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.227005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.180888 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-lib-modules\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.227809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.180909 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/163da31b-27c0-4e33-b898-4a2066ccea13-host\") pod \"node-ca-44h8t\" (UID: \"163da31b-27c0-4e33-b898-4a2066ccea13\") " pod="openshift-image-registry/node-ca-44h8t" Jul 21 18:00:01.227809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.180949 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-etc-selinux\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.227809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181029 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.227809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181057 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-system-cni-dir\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.227809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181086 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-system-cni-dir\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.227809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181101 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-etc-selinux\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.227809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181110 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.227809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181116 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-os-release\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.227809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181163 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-os-release\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.227809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181190 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-kubernetes\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.227809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181205 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.227809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181218 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-tmp\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.227809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181236 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2c224dc7-b84d-4e28-84fe-686e4f9735ac-konnectivity-ca\") pod \"konnectivity-agent-gtv75\" (UID: \"2c224dc7-b84d-4e28-84fe-686e4f9735ac\") " pod="kube-system/konnectivity-agent-gtv75" Jul 21 18:00:01.227809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181252 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmpxp\" (UniqueName: \"kubernetes.io/projected/91e663b2-9fae-459f-bbe2-bcd2a022727a-kube-api-access-gmpxp\") pod \"iptables-alerter-zr6bx\" (UID: \"91e663b2-9fae-459f-bbe2-bcd2a022727a\") " pod="openshift-network-operator/iptables-alerter-zr6bx" Jul 21 18:00:01.228610 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181267 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-run-systemd\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.228610 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181282 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6746d28c-eb2a-40ec-ab6d-da2e086116dc-ovnkube-config\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.228610 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181305 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-var-lib-kubelet\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.228610 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181321 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-run-systemd\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.228610 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181325 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-host\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.228610 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181347 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc216396-5088-4fd3-9409-f525712b3b92-tmp-dir\") pod \"node-resolver-vlkxx\" (UID: \"dc216396-5088-4fd3-9409-f525712b3b92\") " pod="openshift-dns/node-resolver-vlkxx" Jul 21 18:00:01.228610 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181361 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/91e663b2-9fae-459f-bbe2-bcd2a022727a-host-slash\") pod \"iptables-alerter-zr6bx\" (UID: \"91e663b2-9fae-459f-bbe2-bcd2a022727a\") " pod="openshift-network-operator/iptables-alerter-zr6bx" Jul 21 18:00:01.228610 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181393 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-run-ovn\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.228610 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181442 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-run-ovn\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.228610 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181465 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/91e663b2-9fae-459f-bbe2-bcd2a022727a-host-slash\") pod \"iptables-alerter-zr6bx\" (UID: \"91e663b2-9fae-459f-bbe2-bcd2a022727a\") " pod="openshift-network-operator/iptables-alerter-zr6bx" Jul 21 18:00:01.228610 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181485 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6746d28c-eb2a-40ec-ab6d-da2e086116dc-ovn-node-metrics-cert\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.228610 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181529 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5xzl\" (UniqueName: \"kubernetes.io/projected/6746d28c-eb2a-40ec-ab6d-da2e086116dc-kube-api-access-b5xzl\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.228610 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181629 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-multus-socket-dir-parent\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.228610 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181684 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-run-k8s-cni-cncf-io\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.233476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181715 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-multus-cni-dir\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.233476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181739 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7cdc92e-ed55-4606-bd31-6cf648f24460-multus-daemon-config\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.233476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181763 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-run\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.233476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181785 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-kubelet-config\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:01.233476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181824 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:01.233476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181846 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-sys-fs\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.233476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181862 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg47k\" (UniqueName: \"kubernetes.io/projected/dc216396-5088-4fd3-9409-f525712b3b92-kube-api-access-cg47k\") pod \"node-resolver-vlkxx\" (UID: \"dc216396-5088-4fd3-9409-f525712b3b92\") " pod="openshift-dns/node-resolver-vlkxx" Jul 21 18:00:01.233476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181887 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph6pd\" (UniqueName: \"kubernetes.io/projected/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-kube-api-access-ph6pd\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.233476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181914 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-278cw\" (UniqueName: \"kubernetes.io/projected/f7cdc92e-ed55-4606-bd31-6cf648f24460-kube-api-access-278cw\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.233476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181956 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-tuned\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.233476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.181985 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-cni-bin\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.233476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182013 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.233476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182039 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-run-netns\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.233476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182083 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-sysconfig\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182112 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-systemd-units\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182141 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-log-socket\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182168 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7cdc92e-ed55-4606-bd31-6cf648f24460-cni-binary-copy\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182193 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-var-lib-cni-bin\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182218 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-var-lib-kubelet\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182271 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gqgn\" (UniqueName: \"kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn\") pod \"network-check-target-4582q\" (UID: \"6f10df5a-b0d9-49b7-b495-247456b8cdd8\") " pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182305 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2c224dc7-b84d-4e28-84fe-686e4f9735ac-agent-certs\") pod \"konnectivity-agent-gtv75\" (UID: \"2c224dc7-b84d-4e28-84fe-686e4f9735ac\") " pod="kube-system/konnectivity-agent-gtv75" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182353 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-run-openvswitch\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182382 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-kubelet\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182467 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-kubelet\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182468 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f740821c-eba5-4c6c-bf39-f1ab91c70256-sys-fs\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182530 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182542 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-run-openvswitch\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.182556 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-systemd-units\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.234310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.183003 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-host-cni-bin\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.235127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.183020 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-cni-binary-copy\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.235127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.183030 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.235127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.183021 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6746d28c-eb2a-40ec-ab6d-da2e086116dc-log-socket\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.235127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.183458 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6746d28c-eb2a-40ec-ab6d-da2e086116dc-env-overrides\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.235127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.183542 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6746d28c-eb2a-40ec-ab6d-da2e086116dc-ovnkube-script-lib\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.235127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.183560 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6746d28c-eb2a-40ec-ab6d-da2e086116dc-ovnkube-config\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.235127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.184200 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc216396-5088-4fd3-9409-f525712b3b92-tmp-dir\") pod \"node-resolver-vlkxx\" (UID: \"dc216396-5088-4fd3-9409-f525712b3b92\") " pod="openshift-dns/node-resolver-vlkxx" Jul 21 18:00:01.235127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.184289 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/91e663b2-9fae-459f-bbe2-bcd2a022727a-iptables-alerter-script\") pod \"iptables-alerter-zr6bx\" (UID: \"91e663b2-9fae-459f-bbe2-bcd2a022727a\") " pod="openshift-network-operator/iptables-alerter-zr6bx" Jul 21 18:00:01.235127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.184333 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.235127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.186186 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6746d28c-eb2a-40ec-ab6d-da2e086116dc-ovn-node-metrics-cert\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.235127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.188373 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hsjq\" (UniqueName: \"kubernetes.io/projected/f740821c-eba5-4c6c-bf39-f1ab91c70256-kube-api-access-8hsjq\") pod \"aws-ebs-csi-driver-node-nk2lb\" (UID: \"f740821c-eba5-4c6c-bf39-f1ab91c70256\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.235127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.188637 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmhpt\" (UniqueName: \"kubernetes.io/projected/89ffba65-3615-478c-856d-a75629de4606-kube-api-access-vmhpt\") pod \"network-metrics-daemon-kr8ht\" (UID: \"89ffba65-3615-478c-856d-a75629de4606\") " pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:01.235127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.189014 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5xzl\" (UniqueName: \"kubernetes.io/projected/6746d28c-eb2a-40ec-ab6d-da2e086116dc-kube-api-access-b5xzl\") pod \"ovnkube-node-7h7m4\" (UID: \"6746d28c-eb2a-40ec-ab6d-da2e086116dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.235127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.189230 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmpxp\" (UniqueName: \"kubernetes.io/projected/91e663b2-9fae-459f-bbe2-bcd2a022727a-kube-api-access-gmpxp\") pod \"iptables-alerter-zr6bx\" (UID: \"91e663b2-9fae-459f-bbe2-bcd2a022727a\") " pod="openshift-network-operator/iptables-alerter-zr6bx" Jul 21 18:00:01.237952 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.190250 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg47k\" (UniqueName: \"kubernetes.io/projected/dc216396-5088-4fd3-9409-f525712b3b92-kube-api-access-cg47k\") pod \"node-resolver-vlkxx\" (UID: \"dc216396-5088-4fd3-9409-f525712b3b92\") " pod="openshift-dns/node-resolver-vlkxx" Jul 21 18:00:01.237952 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.190431 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph6pd\" (UniqueName: \"kubernetes.io/projected/3257df56-1fd0-4f11-a9fa-b68d6daa1a1b-kube-api-access-ph6pd\") pod \"multus-additional-cni-plugins-q8fqf\" (UID: \"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b\") " pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283200 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-var-lib-cni-multus\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283233 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-sysctl-d\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283252 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-sysctl-conf\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283269 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k9rg\" (UniqueName: \"kubernetes.io/projected/163da31b-27c0-4e33-b898-4a2066ccea13-kube-api-access-2k9rg\") pod \"node-ca-44h8t\" (UID: \"163da31b-27c0-4e33-b898-4a2066ccea13\") " pod="openshift-image-registry/node-ca-44h8t" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283292 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-os-release\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283312 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-multus-conf-dir\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283324 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-var-lib-cni-multus\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283350 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-multus-conf-dir\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283396 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-os-release\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283412 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-sysctl-d\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283421 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-lib-modules\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283436 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-sysctl-conf\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283464 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/163da31b-27c0-4e33-b898-4a2066ccea13-host\") pod \"node-ca-44h8t\" (UID: \"163da31b-27c0-4e33-b898-4a2066ccea13\") " pod="openshift-image-registry/node-ca-44h8t" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283490 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-lib-modules\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.343799 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283495 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-kubernetes\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283516 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-tmp\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283518 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/163da31b-27c0-4e33-b898-4a2066ccea13-host\") pod \"node-ca-44h8t\" (UID: \"163da31b-27c0-4e33-b898-4a2066ccea13\") " pod="openshift-image-registry/node-ca-44h8t" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283540 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2c224dc7-b84d-4e28-84fe-686e4f9735ac-konnectivity-ca\") pod \"konnectivity-agent-gtv75\" (UID: \"2c224dc7-b84d-4e28-84fe-686e4f9735ac\") " pod="kube-system/konnectivity-agent-gtv75" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283560 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-var-lib-kubelet\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283579 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-host\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283601 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-kubernetes\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-multus-socket-dir-parent\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283628 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-var-lib-kubelet\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283648 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-run-k8s-cni-cncf-io\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283654 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-host\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283673 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-multus-cni-dir\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283690 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-run-k8s-cni-cncf-io\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283695 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7cdc92e-ed55-4606-bd31-6cf648f24460-multus-daemon-config\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283714 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-multus-socket-dir-parent\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.449243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283718 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-run\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.450431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283740 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-kubelet-config\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:01.450431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283798 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:01.450431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283810 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-multus-cni-dir\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.450431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283803 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-run\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.450431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283835 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-278cw\" (UniqueName: \"kubernetes.io/projected/f7cdc92e-ed55-4606-bd31-6cf648f24460-kube-api-access-278cw\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.450431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283835 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-kubelet-config\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:01.450431 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.283896 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:01.450431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.283955 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-tuned\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.450431 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.283975 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret podName:f9411eb0-f8da-4954-8df7-8fa2df70b0f0 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:01.783955675 +0000 UTC m=+3.052267113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret") pod "global-pull-secret-syncer-hrk92" (UID: "f9411eb0-f8da-4954-8df7-8fa2df70b0f0") : object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:01.450431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284005 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-run-netns\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.450431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284029 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-sysconfig\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.450431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284065 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7cdc92e-ed55-4606-bd31-6cf648f24460-cni-binary-copy\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.450431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284073 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-run-netns\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.450431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284094 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-var-lib-cni-bin\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284119 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-var-lib-kubelet\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284129 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-sysconfig\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284143 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gqgn\" (UniqueName: \"kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn\") pod \"network-check-target-4582q\" (UID: \"6f10df5a-b0d9-49b7-b495-247456b8cdd8\") " pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284163 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-var-lib-cni-bin\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284168 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2c224dc7-b84d-4e28-84fe-686e4f9735ac-agent-certs\") pod \"konnectivity-agent-gtv75\" (UID: \"2c224dc7-b84d-4e28-84fe-686e4f9735ac\") " pod="kube-system/konnectivity-agent-gtv75" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284224 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-var-lib-kubelet\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284261 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-etc-kubernetes\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284286 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/163da31b-27c0-4e33-b898-4a2066ccea13-serviceca\") pod \"node-ca-44h8t\" (UID: \"163da31b-27c0-4e33-b898-4a2066ccea13\") " pod="openshift-image-registry/node-ca-44h8t" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284316 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-system-cni-dir\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284339 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-systemd\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284365 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-run-multus-certs\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284368 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2c224dc7-b84d-4e28-84fe-686e4f9735ac-konnectivity-ca\") pod \"konnectivity-agent-gtv75\" (UID: \"2c224dc7-b84d-4e28-84fe-686e4f9735ac\") " pod="kube-system/konnectivity-agent-gtv75" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284388 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-sys\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284417 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7546\" (UniqueName: \"kubernetes.io/projected/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-kube-api-access-m7546\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.450859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284437 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-systemd\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284444 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-dbus\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284476 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-etc-kubernetes\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284490 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-dbus\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284515 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-cnibin\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284436 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7cdc92e-ed55-4606-bd31-6cf648f24460-multus-daemon-config\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284476 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-cnibin\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284567 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-modprobe-d\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284561 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-host-run-multus-certs\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284609 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-hostroot\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284617 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-sys\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284657 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-hostroot\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284535 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7cdc92e-ed55-4606-bd31-6cf648f24460-system-cni-dir\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284749 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-modprobe-d\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.284805 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/163da31b-27c0-4e33-b898-4a2066ccea13-serviceca\") pod \"node-ca-44h8t\" (UID: \"163da31b-27c0-4e33-b898-4a2066ccea13\") " pod="openshift-image-registry/node-ca-44h8t" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.285156 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7cdc92e-ed55-4606-bd31-6cf648f24460-cni-binary-copy\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.452449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.286161 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-etc-tuned\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.453283 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.286494 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-tmp\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.453283 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.286650 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2c224dc7-b84d-4e28-84fe-686e4f9735ac-agent-certs\") pod \"konnectivity-agent-gtv75\" (UID: \"2c224dc7-b84d-4e28-84fe-686e4f9735ac\") " pod="kube-system/konnectivity-agent-gtv75" Jul 21 18:00:01.453283 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.293442 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 18:00:01.453283 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.293459 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 18:00:01.453283 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.293471 2568 projected.go:196] Error preparing data for projected volume kube-api-access-5gqgn for pod openshift-network-diagnostics/network-check-target-4582q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:01.453283 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.293506 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-278cw\" (UniqueName: \"kubernetes.io/projected/f7cdc92e-ed55-4606-bd31-6cf648f24460-kube-api-access-278cw\") pod \"multus-bnghg\" (UID: \"f7cdc92e-ed55-4606-bd31-6cf648f24460\") " pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.453283 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.293454 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k9rg\" (UniqueName: \"kubernetes.io/projected/163da31b-27c0-4e33-b898-4a2066ccea13-kube-api-access-2k9rg\") pod \"node-ca-44h8t\" (UID: \"163da31b-27c0-4e33-b898-4a2066ccea13\") " pod="openshift-image-registry/node-ca-44h8t" Jul 21 18:00:01.453283 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.293579 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn podName:6f10df5a-b0d9-49b7-b495-247456b8cdd8 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:01.793565454 +0000 UTC m=+3.061876896 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5gqgn" (UniqueName: "kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn") pod "network-check-target-4582q" (UID: "6f10df5a-b0d9-49b7-b495-247456b8cdd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:01.453283 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.294966 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7546\" (UniqueName: \"kubernetes.io/projected/0f7af5f5-05d1-44e2-aca6-b74f7a5faf20-kube-api-access-m7546\") pod \"tuned-fzz5k\" (UID: \"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20\") " pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.453283 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.403079 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 18:00:01.553826 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.553802 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:01.570807 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.570778 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q8fqf" Jul 21 18:00:01.589058 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.589032 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" Jul 21 18:00:01.605486 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.605467 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vlkxx" Jul 21 18:00:01.626489 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.626465 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zr6bx" Jul 21 18:00:01.641420 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.641396 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" Jul 21 18:00:01.655736 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.655711 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gtv75" Jul 21 18:00:01.661709 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.661689 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-44h8t" Jul 21 18:00:01.678929 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.678906 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bnghg" Jul 21 18:00:01.688712 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.688690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs\") pod \"network-metrics-daemon-kr8ht\" (UID: \"89ffba65-3615-478c-856d-a75629de4606\") " pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:01.688821 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.688792 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:01.688881 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.688869 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs podName:89ffba65-3615-478c-856d-a75629de4606 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:02.688849684 +0000 UTC m=+3.957161124 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs") pod "network-metrics-daemon-kr8ht" (UID: "89ffba65-3615-478c-856d-a75629de4606") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:01.729475 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:01.729442 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7cdc92e_ed55_4606_bd31_6cf648f24460.slice/crio-0bf39a3a81d5b52acdf18572a39b4d1ab554cfc9b1da3c945cbcf48ff1d28423 WatchSource:0}: Error finding container 0bf39a3a81d5b52acdf18572a39b4d1ab554cfc9b1da3c945cbcf48ff1d28423: Status 404 returned error can't find the container with id 0bf39a3a81d5b52acdf18572a39b4d1ab554cfc9b1da3c945cbcf48ff1d28423 Jul 21 18:00:01.729700 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:01.729674 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6d68d762c0ec53c9f113093b050797.slice/crio-bfce70c6bfe223adef74954498b1a3afb735f5f15c3704bebe5ec02dcb4eb369 WatchSource:0}: Error finding container bfce70c6bfe223adef74954498b1a3afb735f5f15c3704bebe5ec02dcb4eb369: Status 404 returned error can't find the container with id bfce70c6bfe223adef74954498b1a3afb735f5f15c3704bebe5ec02dcb4eb369 Jul 21 18:00:01.732077 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:01.732055 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a6fe56b86f91ef9f0ca8f3090eba9bd.slice/crio-7fce4abff33d2b433b8c91d28e8b82d7c23c3fe70395d863b137f55df477497e WatchSource:0}: Error finding container 7fce4abff33d2b433b8c91d28e8b82d7c23c3fe70395d863b137f55df477497e: Status 404 returned error can't find the container with id 7fce4abff33d2b433b8c91d28e8b82d7c23c3fe70395d863b137f55df477497e Jul 21 18:00:01.733255 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.733238 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 18:00:01.790924 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.790884 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:01.791149 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.791050 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:01.791149 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.791129 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret podName:f9411eb0-f8da-4954-8df7-8fa2df70b0f0 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:02.791107177 +0000 UTC m=+4.059418620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret") pod "global-pull-secret-syncer-hrk92" (UID: "f9411eb0-f8da-4954-8df7-8fa2df70b0f0") : object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:01.891682 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:01.891584 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gqgn\" (UniqueName: \"kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn\") pod \"network-check-target-4582q\" (UID: \"6f10df5a-b0d9-49b7-b495-247456b8cdd8\") " pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:01.891841 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.891722 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 18:00:01.891841 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.891739 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 18:00:01.891841 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.891748 2568 projected.go:196] Error preparing data for projected volume kube-api-access-5gqgn for pod openshift-network-diagnostics/network-check-target-4582q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:01.891841 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:01.891798 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn podName:6f10df5a-b0d9-49b7-b495-247456b8cdd8 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:02.891784635 +0000 UTC m=+4.160096068 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5gqgn" (UniqueName: "kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn") pod "network-check-target-4582q" (UID: "6f10df5a-b0d9-49b7-b495-247456b8cdd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:02.132829 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:02.132775 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 17:55:01 +0000 UTC" deadline="2028-03-26 05:01:47.714982687 +0000 UTC" Jul 21 18:00:02.132829 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:02.132820 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14723h1m45.582166789s" Jul 21 18:00:02.212561 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:02.212482 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:02.212702 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:02.212589 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:02.219405 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:02.219353 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-17.ec2.internal" event={"ID":"0a6fe56b86f91ef9f0ca8f3090eba9bd","Type":"ContainerStarted","Data":"7fce4abff33d2b433b8c91d28e8b82d7c23c3fe70395d863b137f55df477497e"} Jul 21 18:00:02.220417 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:02.220393 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal" event={"ID":"ac6d68d762c0ec53c9f113093b050797","Type":"ContainerStarted","Data":"bfce70c6bfe223adef74954498b1a3afb735f5f15c3704bebe5ec02dcb4eb369"} Jul 21 18:00:02.221554 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:02.221527 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bnghg" event={"ID":"f7cdc92e-ed55-4606-bd31-6cf648f24460","Type":"ContainerStarted","Data":"0bf39a3a81d5b52acdf18572a39b4d1ab554cfc9b1da3c945cbcf48ff1d28423"} Jul 21 18:00:02.451661 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:02.451632 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf740821c_eba5_4c6c_bf39_f1ab91c70256.slice/crio-706d81edbade8e05ce9e848c0e1d2b8b8827677e4f8a82ba9ee3094fc990ba85 WatchSource:0}: Error finding container 706d81edbade8e05ce9e848c0e1d2b8b8827677e4f8a82ba9ee3094fc990ba85: Status 404 returned error can't find the container with id 706d81edbade8e05ce9e848c0e1d2b8b8827677e4f8a82ba9ee3094fc990ba85 Jul 21 18:00:02.472609 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:02.472577 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6746d28c_eb2a_40ec_ab6d_da2e086116dc.slice/crio-ed953b2752bc8e026347480d14b9b304f2da274c3aab1b279aefe754d27ffa84 WatchSource:0}: Error finding container ed953b2752bc8e026347480d14b9b304f2da274c3aab1b279aefe754d27ffa84: Status 404 returned error can't find the container with id ed953b2752bc8e026347480d14b9b304f2da274c3aab1b279aefe754d27ffa84 Jul 21 18:00:02.543391 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:02.543350 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc216396_5088_4fd3_9409_f525712b3b92.slice/crio-168900957c15706993e4d964c29df1d8de21cfc2afa358e2486600c7876da517 WatchSource:0}: Error finding container 168900957c15706993e4d964c29df1d8de21cfc2afa358e2486600c7876da517: Status 404 returned error can't find the container with id 168900957c15706993e4d964c29df1d8de21cfc2afa358e2486600c7876da517 Jul 21 18:00:02.566216 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:02.566186 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c224dc7_b84d_4e28_84fe_686e4f9735ac.slice/crio-17ce95d1c2bd69206b5ea46a6a0fcc87334844c621d0427d14a6b479bfc07de9 WatchSource:0}: Error finding container 17ce95d1c2bd69206b5ea46a6a0fcc87334844c621d0427d14a6b479bfc07de9: Status 404 returned error can't find the container with id 17ce95d1c2bd69206b5ea46a6a0fcc87334844c621d0427d14a6b479bfc07de9 Jul 21 18:00:02.642149 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:02.642114 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91e663b2_9fae_459f_bbe2_bcd2a022727a.slice/crio-27a148e353b1b3538384a4f535bd3f6ea142b6cb13f4715b5538dd4fdc1e756c WatchSource:0}: Error finding container 27a148e353b1b3538384a4f535bd3f6ea142b6cb13f4715b5538dd4fdc1e756c: Status 404 returned error can't find the container with id 27a148e353b1b3538384a4f535bd3f6ea142b6cb13f4715b5538dd4fdc1e756c Jul 21 18:00:02.661134 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:02.661104 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3257df56_1fd0_4f11_a9fa_b68d6daa1a1b.slice/crio-c2f78f044a2e737c4f62177b7d6d2bb16970989e03af2654c567b6d02e91d869 WatchSource:0}: Error finding container c2f78f044a2e737c4f62177b7d6d2bb16970989e03af2654c567b6d02e91d869: Status 404 returned error can't find the container with id c2f78f044a2e737c4f62177b7d6d2bb16970989e03af2654c567b6d02e91d869 Jul 21 18:00:02.696566 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:02.696527 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs\") pod \"network-metrics-daemon-kr8ht\" (UID: \"89ffba65-3615-478c-856d-a75629de4606\") " pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:02.696778 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:02.696754 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:02.696875 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:02.696835 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs podName:89ffba65-3615-478c-856d-a75629de4606 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:04.69681291 +0000 UTC m=+5.965124349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs") pod "network-metrics-daemon-kr8ht" (UID: "89ffba65-3615-478c-856d-a75629de4606") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:02.705586 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:02.705562 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f7af5f5_05d1_44e2_aca6_b74f7a5faf20.slice/crio-2ec38ce24964729efe4613e150c4d740d67a2b232555b4b1c4995bb430890d8e WatchSource:0}: Error finding container 2ec38ce24964729efe4613e150c4d740d67a2b232555b4b1c4995bb430890d8e: Status 404 returned error can't find the container with id 2ec38ce24964729efe4613e150c4d740d67a2b232555b4b1c4995bb430890d8e Jul 21 18:00:02.797974 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:02.797881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:02.798109 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:02.798065 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:02.798148 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:02.798134 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret podName:f9411eb0-f8da-4954-8df7-8fa2df70b0f0 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:04.798119304 +0000 UTC m=+6.066430724 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret") pod "global-pull-secret-syncer-hrk92" (UID: "f9411eb0-f8da-4954-8df7-8fa2df70b0f0") : object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:02.871411 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:02.871376 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod163da31b_27c0_4e33_b898_4a2066ccea13.slice/crio-c4b95e150a935bc963130ea56eda5f7f6af93829f687fce90291a061657071ed WatchSource:0}: Error finding container c4b95e150a935bc963130ea56eda5f7f6af93829f687fce90291a061657071ed: Status 404 returned error can't find the container with id c4b95e150a935bc963130ea56eda5f7f6af93829f687fce90291a061657071ed Jul 21 18:00:02.875289 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:02.875263 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 18:00:02.899013 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:02.898976 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gqgn\" (UniqueName: \"kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn\") pod \"network-check-target-4582q\" (UID: \"6f10df5a-b0d9-49b7-b495-247456b8cdd8\") " pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:02.899199 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:02.899142 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 18:00:02.899199 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:02.899163 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 18:00:02.899199 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:02.899176 2568 projected.go:196] Error preparing data for projected volume kube-api-access-5gqgn for pod openshift-network-diagnostics/network-check-target-4582q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:02.899379 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:02.899242 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn podName:6f10df5a-b0d9-49b7-b495-247456b8cdd8 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:04.899222201 +0000 UTC m=+6.167533621 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5gqgn" (UniqueName: "kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn") pod "network-check-target-4582q" (UID: "6f10df5a-b0d9-49b7-b495-247456b8cdd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:03.133346 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:03.133246 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 17:55:01 +0000 UTC" deadline="2028-01-03 01:51:53.554496675 +0000 UTC" Jul 21 18:00:03.133346 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:03.133288 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12727h51m50.421214852s" Jul 21 18:00:03.212581 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:03.212547 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:03.212754 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:03.212717 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:03.212823 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:03.212785 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:03.212967 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:03.212921 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:03.224566 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:03.224531 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" event={"ID":"f740821c-eba5-4c6c-bf39-f1ab91c70256","Type":"ContainerStarted","Data":"706d81edbade8e05ce9e848c0e1d2b8b8827677e4f8a82ba9ee3094fc990ba85"} Jul 21 18:00:03.226125 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:03.226057 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-44h8t" event={"ID":"163da31b-27c0-4e33-b898-4a2066ccea13","Type":"ContainerStarted","Data":"c4b95e150a935bc963130ea56eda5f7f6af93829f687fce90291a061657071ed"} Jul 21 18:00:03.227709 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:03.227682 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" event={"ID":"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20","Type":"ContainerStarted","Data":"2ec38ce24964729efe4613e150c4d740d67a2b232555b4b1c4995bb430890d8e"} Jul 21 18:00:03.229068 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:03.229018 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8fqf" event={"ID":"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b","Type":"ContainerStarted","Data":"c2f78f044a2e737c4f62177b7d6d2bb16970989e03af2654c567b6d02e91d869"} Jul 21 18:00:03.230382 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:03.230357 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zr6bx" event={"ID":"91e663b2-9fae-459f-bbe2-bcd2a022727a","Type":"ContainerStarted","Data":"27a148e353b1b3538384a4f535bd3f6ea142b6cb13f4715b5538dd4fdc1e756c"} Jul 21 18:00:03.232024 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:03.231993 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vlkxx" event={"ID":"dc216396-5088-4fd3-9409-f525712b3b92","Type":"ContainerStarted","Data":"168900957c15706993e4d964c29df1d8de21cfc2afa358e2486600c7876da517"} Jul 21 18:00:03.233312 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:03.233285 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gtv75" event={"ID":"2c224dc7-b84d-4e28-84fe-686e4f9735ac","Type":"ContainerStarted","Data":"17ce95d1c2bd69206b5ea46a6a0fcc87334844c621d0427d14a6b479bfc07de9"} Jul 21 18:00:03.234775 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:03.234748 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" event={"ID":"6746d28c-eb2a-40ec-ab6d-da2e086116dc","Type":"ContainerStarted","Data":"ed953b2752bc8e026347480d14b9b304f2da274c3aab1b279aefe754d27ffa84"} Jul 21 18:00:04.212561 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:04.212478 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:04.212722 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:04.212614 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:04.260866 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:04.260829 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-17.ec2.internal" event={"ID":"0a6fe56b86f91ef9f0ca8f3090eba9bd","Type":"ContainerStarted","Data":"b76f6f7e0ee7a87cf81fed04a070178d1ab67eaf1cd46c71d9c860423d774b57"} Jul 21 18:00:04.713024 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:04.712978 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs\") pod \"network-metrics-daemon-kr8ht\" (UID: \"89ffba65-3615-478c-856d-a75629de4606\") " pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:04.713226 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:04.713208 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:04.713309 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:04.713277 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs podName:89ffba65-3615-478c-856d-a75629de4606 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:08.713256864 +0000 UTC m=+9.981568304 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs") pod "network-metrics-daemon-kr8ht" (UID: "89ffba65-3615-478c-856d-a75629de4606") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:04.814085 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:04.814046 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:04.814255 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:04.814228 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:04.814331 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:04.814299 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret podName:f9411eb0-f8da-4954-8df7-8fa2df70b0f0 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:08.814279182 +0000 UTC m=+10.082590626 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret") pod "global-pull-secret-syncer-hrk92" (UID: "f9411eb0-f8da-4954-8df7-8fa2df70b0f0") : object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:04.915310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:04.915268 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gqgn\" (UniqueName: \"kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn\") pod \"network-check-target-4582q\" (UID: \"6f10df5a-b0d9-49b7-b495-247456b8cdd8\") " pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:04.915484 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:04.915442 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 18:00:04.915484 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:04.915465 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 18:00:04.915484 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:04.915477 2568 projected.go:196] Error preparing data for projected volume kube-api-access-5gqgn for pod openshift-network-diagnostics/network-check-target-4582q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:04.915660 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:04.915539 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn podName:6f10df5a-b0d9-49b7-b495-247456b8cdd8 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:08.915521719 +0000 UTC m=+10.183833156 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5gqgn" (UniqueName: "kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn") pod "network-check-target-4582q" (UID: "6f10df5a-b0d9-49b7-b495-247456b8cdd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:05.216166 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:05.215860 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:05.216345 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:05.216254 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:05.216665 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:05.216643 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:05.216843 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:05.216759 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:06.212634 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:06.212597 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:06.213137 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:06.212743 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:07.213387 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:07.213355 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:07.213852 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:07.213357 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:07.213852 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:07.213499 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:07.213852 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:07.213591 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:08.212739 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:08.212667 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:08.212925 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:08.212847 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:08.747843 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:08.747803 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs\") pod \"network-metrics-daemon-kr8ht\" (UID: \"89ffba65-3615-478c-856d-a75629de4606\") " pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:08.748338 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:08.748010 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:08.748338 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:08.748072 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs podName:89ffba65-3615-478c-856d-a75629de4606 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:16.748053986 +0000 UTC m=+18.016365413 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs") pod "network-metrics-daemon-kr8ht" (UID: "89ffba65-3615-478c-856d-a75629de4606") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:08.848836 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:08.848802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:08.849043 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:08.848954 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:08.849043 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:08.849027 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret podName:f9411eb0-f8da-4954-8df7-8fa2df70b0f0 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:16.849006412 +0000 UTC m=+18.117317859 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret") pod "global-pull-secret-syncer-hrk92" (UID: "f9411eb0-f8da-4954-8df7-8fa2df70b0f0") : object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:08.949403 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:08.949348 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gqgn\" (UniqueName: \"kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn\") pod \"network-check-target-4582q\" (UID: \"6f10df5a-b0d9-49b7-b495-247456b8cdd8\") " pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:08.949603 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:08.949553 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 18:00:08.949603 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:08.949575 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 18:00:08.949603 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:08.949586 2568 projected.go:196] Error preparing data for projected volume kube-api-access-5gqgn for pod openshift-network-diagnostics/network-check-target-4582q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:08.949735 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:08.949639 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn podName:6f10df5a-b0d9-49b7-b495-247456b8cdd8 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:16.949625302 +0000 UTC m=+18.217936737 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5gqgn" (UniqueName: "kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn") pod "network-check-target-4582q" (UID: "6f10df5a-b0d9-49b7-b495-247456b8cdd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:09.212577 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:09.212484 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:09.212577 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:09.212528 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:09.212833 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:09.212626 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:09.212833 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:09.212713 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:10.213590 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:10.213419 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:10.213590 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:10.213570 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:11.213642 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:11.213603 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:11.213642 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:11.213626 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:11.214191 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:11.213749 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:11.214191 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:11.213869 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:12.212498 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:12.212459 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:12.212684 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:12.212598 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:13.212914 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:13.212876 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:13.213394 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:13.212887 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:13.213394 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:13.213025 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:13.213394 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:13.213098 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:14.212953 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:14.212861 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:14.213364 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:14.212999 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:15.213450 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:15.213413 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:15.213884 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:15.213413 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:15.213884 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:15.213576 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:15.213884 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:15.213629 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:16.213246 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:16.213210 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:16.213447 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:16.213335 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:16.814212 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:16.814171 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs\") pod \"network-metrics-daemon-kr8ht\" (UID: \"89ffba65-3615-478c-856d-a75629de4606\") " pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:16.814646 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:16.814309 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:16.814646 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:16.814388 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs podName:89ffba65-3615-478c-856d-a75629de4606 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:32.81436627 +0000 UTC m=+34.082677714 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs") pod "network-metrics-daemon-kr8ht" (UID: "89ffba65-3615-478c-856d-a75629de4606") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:16.915235 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:16.915195 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:16.915412 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:16.915355 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:16.915470 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:16.915436 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret podName:f9411eb0-f8da-4954-8df7-8fa2df70b0f0 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:32.915415058 +0000 UTC m=+34.183726498 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret") pod "global-pull-secret-syncer-hrk92" (UID: "f9411eb0-f8da-4954-8df7-8fa2df70b0f0") : object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:17.015573 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:17.015542 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gqgn\" (UniqueName: \"kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn\") pod \"network-check-target-4582q\" (UID: \"6f10df5a-b0d9-49b7-b495-247456b8cdd8\") " pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:17.015751 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:17.015733 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 18:00:17.015813 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:17.015758 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 18:00:17.015813 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:17.015771 2568 projected.go:196] Error preparing data for projected volume kube-api-access-5gqgn for pod openshift-network-diagnostics/network-check-target-4582q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:17.015996 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:17.015836 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn podName:6f10df5a-b0d9-49b7-b495-247456b8cdd8 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:33.015817535 +0000 UTC m=+34.284128978 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5gqgn" (UniqueName: "kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn") pod "network-check-target-4582q" (UID: "6f10df5a-b0d9-49b7-b495-247456b8cdd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:17.212804 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:17.212770 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:17.213028 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:17.212926 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:17.213028 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:17.212982 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:17.213142 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:17.213109 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:18.212821 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:18.212727 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:18.213285 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:18.212839 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:19.215766 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:19.215730 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:19.215766 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:19.215747 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:19.216286 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:19.215852 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:19.216286 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:19.216008 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:20.212898 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:20.212862 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:20.213098 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:20.212984 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:21.213529 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:21.213495 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:21.214024 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:21.213495 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:21.214024 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:21.213647 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:21.214024 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:21.213683 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:22.213251 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:22.213221 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:22.213380 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:22.213332 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:23.212990 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.212560 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:23.213915 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.212613 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:23.213915 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:23.213090 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:23.213915 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:23.213184 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:23.306718 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.306682 2568 generic.go:356] "Generic (PLEG): container finished" podID="3257df56-1fd0-4f11-a9fa-b68d6daa1a1b" containerID="84f85ae9f7b33987fbe34aec2c3e2cf6e84de8165f89e8571bce3af4d20648f1" exitCode=0 Jul 21 18:00:23.306913 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.306765 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8fqf" event={"ID":"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b","Type":"ContainerDied","Data":"84f85ae9f7b33987fbe34aec2c3e2cf6e84de8165f89e8571bce3af4d20648f1"} Jul 21 18:00:23.308374 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.308347 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vlkxx" event={"ID":"dc216396-5088-4fd3-9409-f525712b3b92","Type":"ContainerStarted","Data":"1c879cfc841f2ed13989c8efe92cd31ac9b4ee1cb10c17c7163a3ac92805b62d"} Jul 21 18:00:23.310137 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.310037 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gtv75" event={"ID":"2c224dc7-b84d-4e28-84fe-686e4f9735ac","Type":"ContainerStarted","Data":"f39ae0dd93cc0813c92539f04a40e1d2af3c13ca602008021a3368b2252f971b"} Jul 21 18:00:23.313193 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.313150 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:00:23.313523 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.313501 2568 generic.go:356] "Generic (PLEG): container finished" podID="6746d28c-eb2a-40ec-ab6d-da2e086116dc" containerID="022161371cc3c8b42655b39a38e5490daa03c36f0e6fd9ddf6fdd6a2bd5ce956" exitCode=1 Jul 21 18:00:23.313620 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.313573 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" event={"ID":"6746d28c-eb2a-40ec-ab6d-da2e086116dc","Type":"ContainerStarted","Data":"cc9303e2a7b6f0c0e9a080cbd4993245ac4b32afc63dc97b82f417c9228faad9"} Jul 21 18:00:23.313682 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.313660 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" event={"ID":"6746d28c-eb2a-40ec-ab6d-da2e086116dc","Type":"ContainerStarted","Data":"adb377bcc7eb8cb9ff5591342ff13a28737b54f85e18806fc4c3819f33c7d71a"} Jul 21 18:00:23.313740 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.313712 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" event={"ID":"6746d28c-eb2a-40ec-ab6d-da2e086116dc","Type":"ContainerStarted","Data":"5091138fe67dd808fc875693bd5d14364aac5ced209664bb2d2bfe30b529827b"} Jul 21 18:00:23.313790 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.313758 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" event={"ID":"6746d28c-eb2a-40ec-ab6d-da2e086116dc","Type":"ContainerStarted","Data":"3602ce0ebce1f4614864599fac5bc531e061b57c4cffb31e19b28c503f3b2f8e"} Jul 21 18:00:23.313841 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.313800 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" event={"ID":"6746d28c-eb2a-40ec-ab6d-da2e086116dc","Type":"ContainerDied","Data":"022161371cc3c8b42655b39a38e5490daa03c36f0e6fd9ddf6fdd6a2bd5ce956"} Jul 21 18:00:23.313841 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.313833 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" event={"ID":"6746d28c-eb2a-40ec-ab6d-da2e086116dc","Type":"ContainerStarted","Data":"0ea6e627db8cb136ecc7c7174d1cea69b3e8ec9fc70ec312b017722fd08ee761"} Jul 21 18:00:23.315137 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.315113 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" event={"ID":"f740821c-eba5-4c6c-bf39-f1ab91c70256","Type":"ContainerStarted","Data":"06e2ac9b0d885468cfa57fa6f674c2dab901ac76e721b8d407d65a5cdcf55b98"} Jul 21 18:00:23.316725 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.316703 2568 generic.go:356] "Generic (PLEG): container finished" podID="ac6d68d762c0ec53c9f113093b050797" containerID="073f7d98586314686a1ac5edaea4e017cb106ec8e41f04dca3f59db7eae83e25" exitCode=0 Jul 21 18:00:23.316841 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.316786 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal" event={"ID":"ac6d68d762c0ec53c9f113093b050797","Type":"ContainerDied","Data":"073f7d98586314686a1ac5edaea4e017cb106ec8e41f04dca3f59db7eae83e25"} Jul 21 18:00:23.318496 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.318473 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bnghg" event={"ID":"f7cdc92e-ed55-4606-bd31-6cf648f24460","Type":"ContainerStarted","Data":"6d63c7eaf156b02d1ecaac34c7e83e5b0e025cdf612b34a956a2511ecb82c460"} Jul 21 18:00:23.320243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.320200 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-44h8t" event={"ID":"163da31b-27c0-4e33-b898-4a2066ccea13","Type":"ContainerStarted","Data":"34181e9c7a5c8e29176305ca1b28aad97cd8cb09576289e30c4e3e49a3d79905"} Jul 21 18:00:23.321830 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.321811 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" event={"ID":"0f7af5f5-05d1-44e2-aca6-b74f7a5faf20","Type":"ContainerStarted","Data":"c7edc2a405bae2a38880a508299771b9bde431d158422b0c57b2fd312f5d156d"} Jul 21 18:00:23.383463 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.383406 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-17.ec2.internal" podStartSLOduration=23.383390014 podStartE2EDuration="23.383390014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:04.277845139 +0000 UTC m=+5.546156581" watchObservedRunningTime="2026-07-21 18:00:23.383390014 +0000 UTC m=+24.651701455" Jul 21 18:00:23.428156 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.428110 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gtv75" podStartSLOduration=4.8678941479999995 podStartE2EDuration="24.428092727s" totalImagesPullingTime="19.560198579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:23.427639166 +0000 UTC m=+24.695950609" watchObservedRunningTime="2026-07-21 18:00:23.428092727 +0000 UTC m=+24.696404168" Jul 21 18:00:23.448856 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.448812 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bnghg" podStartSLOduration=3.984694304 podStartE2EDuration="24.448795249s" totalImagesPullingTime="20.464100945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:23.448280688 +0000 UTC m=+24.716592139" watchObservedRunningTime="2026-07-21 18:00:23.448795249 +0000 UTC m=+24.717106690" Jul 21 18:00:23.467044 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.466958 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vlkxx" podStartSLOduration=4.85264288 podStartE2EDuration="24.466922567s" totalImagesPullingTime="19.614279687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:23.466783281 +0000 UTC m=+24.735094726" watchObservedRunningTime="2026-07-21 18:00:23.466922567 +0000 UTC m=+24.735234008" Jul 21 18:00:23.488312 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.488264 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fzz5k" podStartSLOduration=5.03406613 podStartE2EDuration="24.488249414s" totalImagesPullingTime="19.454183284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:23.487527118 +0000 UTC m=+24.755838561" watchObservedRunningTime="2026-07-21 18:00:23.488249414 +0000 UTC m=+24.756560856" Jul 21 18:00:23.513471 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.513414 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-44h8t" podStartSLOduration=9.813003936 podStartE2EDuration="24.513399652s" totalImagesPullingTime="14.700395716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:23.509869201 +0000 UTC m=+24.778180643" watchObservedRunningTime="2026-07-21 18:00:23.513399652 +0000 UTC m=+24.781711093" Jul 21 18:00:23.962922 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:23.962893 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 21 18:00:24.211997 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:24.211622 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T18:00:23.962910996Z","UUID":"46033597-0e1d-44c2-88e2-497451647bb0","Handler":null,"Name":"","Endpoint":""} Jul 21 18:00:24.212559 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:24.212451 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:24.212713 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:24.212619 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:24.213513 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:24.213493 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 21 18:00:24.213991 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:24.213536 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 21 18:00:24.326579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:24.326535 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal" event={"ID":"ac6d68d762c0ec53c9f113093b050797","Type":"ContainerStarted","Data":"7123fd261fae1e6c7d0314d27e522a1c333732a0b168a70df0d821c8b52fbe85"} Jul 21 18:00:24.328159 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:24.328117 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zr6bx" event={"ID":"91e663b2-9fae-459f-bbe2-bcd2a022727a","Type":"ContainerStarted","Data":"7fdce9762da0f82109abaed400ae989e162e85fa12d46dee08b347de4202f714"} Jul 21 18:00:24.330339 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:24.330292 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" event={"ID":"f740821c-eba5-4c6c-bf39-f1ab91c70256","Type":"ContainerStarted","Data":"946c71755440981ecb6898dd4e8199b71ef1846539d321447b8fe269c66c6294"} Jul 21 18:00:24.343698 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:24.343645 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-17.ec2.internal" podStartSLOduration=24.343627469 podStartE2EDuration="24.343627469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:24.343502744 +0000 UTC m=+25.611814187" watchObservedRunningTime="2026-07-21 18:00:24.343627469 +0000 UTC m=+25.611938912" Jul 21 18:00:24.358143 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:24.358085 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zr6bx" podStartSLOduration=5.846339371 podStartE2EDuration="25.358060958s" totalImagesPullingTime="19.511721587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:24.358043123 +0000 UTC m=+25.626354593" watchObservedRunningTime="2026-07-21 18:00:24.358060958 +0000 UTC m=+25.626372401" Jul 21 18:00:25.213664 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:25.213639 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:25.214039 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:25.213780 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:25.214039 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:25.214016 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:25.214171 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:25.214126 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:25.337060 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:25.337033 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:00:25.337509 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:25.337481 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" event={"ID":"6746d28c-eb2a-40ec-ab6d-da2e086116dc","Type":"ContainerStarted","Data":"54bd28774f44108dd3558ad6d963bca63a6f9755571f6b025a37d89e2c10764a"} Jul 21 18:00:25.339928 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:25.339840 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" event={"ID":"f740821c-eba5-4c6c-bf39-f1ab91c70256","Type":"ContainerStarted","Data":"671aa8190ed2088c64a80130d5822bfd186afa0e7e19da857dc244ace9798b99"} Jul 21 18:00:26.212691 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:26.212651 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:26.212894 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:26.212791 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:27.096247 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:27.096060 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gtv75" Jul 21 18:00:27.096704 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:27.096663 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gtv75" Jul 21 18:00:27.117440 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:27.117376 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nk2lb" podStartSLOduration=5.953675018 podStartE2EDuration="28.117353927s" totalImagesPullingTime="22.163678909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:59:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:25.371350477 +0000 UTC m=+26.639661918" watchObservedRunningTime="2026-07-21 18:00:27.117353927 +0000 UTC m=+28.385665384" Jul 21 18:00:27.212947 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:27.212891 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:27.213140 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:27.212904 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:27.213140 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:27.213068 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:27.213265 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:27.213133 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:28.213630 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:28.213369 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:28.214295 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:28.213633 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:28.347420 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:28.347385 2568 generic.go:356] "Generic (PLEG): container finished" podID="3257df56-1fd0-4f11-a9fa-b68d6daa1a1b" containerID="82086fc7b6f0cca442524f93581fd8e2de359f0d09eebb90a53a516684e6579a" exitCode=0 Jul 21 18:00:28.347586 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:28.347445 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8fqf" event={"ID":"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b","Type":"ContainerDied","Data":"82086fc7b6f0cca442524f93581fd8e2de359f0d09eebb90a53a516684e6579a"} Jul 21 18:00:28.350986 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:28.350969 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:00:28.351309 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:28.351287 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" event={"ID":"6746d28c-eb2a-40ec-ab6d-da2e086116dc","Type":"ContainerStarted","Data":"42f56240d0488633ac8782a8ae57ba6f9517d1f9628dbac9f4b8881acb627f4c"} Jul 21 18:00:28.351534 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:28.351514 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:28.351600 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:28.351572 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:28.351693 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:28.351678 2568 scope.go:117] "RemoveContainer" containerID="022161371cc3c8b42655b39a38e5490daa03c36f0e6fd9ddf6fdd6a2bd5ce956" Jul 21 18:00:28.367584 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:28.367562 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:29.214625 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:29.214590 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:29.215160 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:29.214722 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:29.215160 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:29.214779 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:29.215160 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:29.214915 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:29.312900 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:29.312857 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gtv75" Jul 21 18:00:29.313131 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:29.313052 2568 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 21 18:00:29.313514 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:29.313496 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gtv75" Jul 21 18:00:29.356965 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:29.356921 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:00:29.357366 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:29.357343 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" event={"ID":"6746d28c-eb2a-40ec-ab6d-da2e086116dc","Type":"ContainerStarted","Data":"7e9adcc3ad351c8cdd40d8c8fce75487ed98b2f60fb52917ba68f60131752e90"} Jul 21 18:00:29.357539 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:29.357525 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:29.373314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:29.373288 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:00:29.500342 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:29.500225 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" podStartSLOduration=10.649206266 podStartE2EDuration="30.500201806s" totalImagesPullingTime="19.85099554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:29.450972614 +0000 UTC m=+30.719284055" watchObservedRunningTime="2026-07-21 18:00:29.500201806 +0000 UTC m=+30.768513248" Jul 21 18:00:30.213367 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:30.213220 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:30.213367 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:30.213310 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:30.264120 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:30.264082 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hrk92"] Jul 21 18:00:30.264554 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:30.264200 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:30.264554 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:30.264279 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:30.268431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:30.268404 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4582q"] Jul 21 18:00:30.275269 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:30.275241 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kr8ht"] Jul 21 18:00:30.275397 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:30.275351 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:30.275454 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:30.275436 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:30.362163 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:30.362132 2568 generic.go:356] "Generic (PLEG): container finished" podID="3257df56-1fd0-4f11-a9fa-b68d6daa1a1b" containerID="443683bfc0d1e5939fb6df199bc0c87e8c6b564a5e6e29bed7d0e86ff4c83baa" exitCode=0 Jul 21 18:00:30.362345 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:30.362210 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:30.362345 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:30.362201 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8fqf" event={"ID":"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b","Type":"ContainerDied","Data":"443683bfc0d1e5939fb6df199bc0c87e8c6b564a5e6e29bed7d0e86ff4c83baa"} Jul 21 18:00:30.362345 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:30.362305 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:32.213262 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:32.213066 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:32.213641 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:32.213066 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:32.213641 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:32.213354 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:32.213641 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:32.213066 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:32.213641 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:32.213411 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:32.213641 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:32.213490 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:32.370126 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:32.370094 2568 generic.go:356] "Generic (PLEG): container finished" podID="3257df56-1fd0-4f11-a9fa-b68d6daa1a1b" containerID="ce19cc76411db0b78f54cc6eea3207003b1bf6b51c29c87eab6b28030bef3dd5" exitCode=0 Jul 21 18:00:32.370298 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:32.370137 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8fqf" event={"ID":"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b","Type":"ContainerDied","Data":"ce19cc76411db0b78f54cc6eea3207003b1bf6b51c29c87eab6b28030bef3dd5"} Jul 21 18:00:32.828065 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:32.828028 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs\") pod \"network-metrics-daemon-kr8ht\" (UID: \"89ffba65-3615-478c-856d-a75629de4606\") " pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:32.828245 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:32.828188 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:32.828292 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:32.828255 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs podName:89ffba65-3615-478c-856d-a75629de4606 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:04.828237984 +0000 UTC m=+66.096549403 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs") pod "network-metrics-daemon-kr8ht" (UID: "89ffba65-3615-478c-856d-a75629de4606") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 18:00:32.929412 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:32.929372 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:32.929646 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:32.929499 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:32.929646 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:32.929561 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret podName:f9411eb0-f8da-4954-8df7-8fa2df70b0f0 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:04.92954353 +0000 UTC m=+66.197854968 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret") pod "global-pull-secret-syncer-hrk92" (UID: "f9411eb0-f8da-4954-8df7-8fa2df70b0f0") : object "kube-system"/"original-pull-secret" not registered Jul 21 18:00:33.029985 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:33.029931 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gqgn\" (UniqueName: \"kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn\") pod \"network-check-target-4582q\" (UID: \"6f10df5a-b0d9-49b7-b495-247456b8cdd8\") " pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:33.030176 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:33.030137 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 18:00:33.030176 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:33.030164 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 18:00:33.030293 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:33.030179 2568 projected.go:196] Error preparing data for projected volume kube-api-access-5gqgn for pod openshift-network-diagnostics/network-check-target-4582q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:33.030293 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:33.030252 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn podName:6f10df5a-b0d9-49b7-b495-247456b8cdd8 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:05.030230885 +0000 UTC m=+66.298542324 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5gqgn" (UniqueName: "kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn") pod "network-check-target-4582q" (UID: "6f10df5a-b0d9-49b7-b495-247456b8cdd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 18:00:34.213303 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:34.213255 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:34.213748 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:34.213374 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:34.213748 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:34.213383 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hrk92" podUID="f9411eb0-f8da-4954-8df7-8fa2df70b0f0" Jul 21 18:00:34.213748 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:34.213411 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:34.213748 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:34.213520 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kr8ht" podUID="89ffba65-3615-478c-856d-a75629de4606" Jul 21 18:00:34.213748 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:34.213597 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4582q" podUID="6f10df5a-b0d9-49b7-b495-247456b8cdd8" Jul 21 18:00:35.087685 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.087613 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-17.ec2.internal" event="NodeReady" Jul 21 18:00:35.087845 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.087759 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 21 18:00:35.154340 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.154308 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6774fd47f-flh6l"] Jul 21 18:00:35.156492 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.156454 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-799cfd6768-8vvdp"] Jul 21 18:00:35.156643 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.156605 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.158386 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.158363 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.167689 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.167664 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 21 18:00:35.167843 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.167664 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 21 18:00:35.168542 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.168384 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 21 18:00:35.168542 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.168417 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 21 18:00:35.168542 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.168441 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 21 18:00:35.168542 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.168463 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 21 18:00:35.168803 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.168543 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 21 18:00:35.168803 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.168719 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 21 18:00:35.168894 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.168804 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7t5gz\"" Jul 21 18:00:35.169046 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.169030 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xvdbv\"" Jul 21 18:00:35.169553 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.169534 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 21 18:00:35.176755 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.176719 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 21 18:00:35.183754 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.183726 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-ztxpr"] Jul 21 18:00:35.186571 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.186547 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-799cfd6768-8vvdp"] Jul 21 18:00:35.186571 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.186586 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf"] Jul 21 18:00:35.186769 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.186702 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.189880 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.189858 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6jjkh"] Jul 21 18:00:35.190063 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.190046 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" Jul 21 18:00:35.191125 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.191105 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 21 18:00:35.191775 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.191756 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7"] Jul 21 18:00:35.191908 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.191891 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6jjkh" Jul 21 18:00:35.193953 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.193569 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 21 18:00:35.197016 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.195876 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 21 18:00:35.197221 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.197103 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq"] Jul 21 18:00:35.197304 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.197271 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 21 18:00:35.198175 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.198157 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:00:35.198351 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.198334 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-lq89g\"" Jul 21 18:00:35.199109 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.199083 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-krmfj"] Jul 21 18:00:35.199341 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.199197 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" Jul 21 18:00:35.200279 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.200260 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 21 18:00:35.201125 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.201106 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-74hdc"] Jul 21 18:00:35.201261 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.201243 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 21 18:00:35.201323 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.201181 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:35.204765 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.204298 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:35.208992 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.208716 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 21 18:00:35.210053 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.210036 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ktdnb\"" Jul 21 18:00:35.210192 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.210173 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 21 18:00:35.211852 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.211755 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6774fd47f-flh6l"] Jul 21 18:00:35.213774 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.213756 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 21 18:00:35.214926 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.214471 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bzs9r\"" Jul 21 18:00:35.214926 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.214514 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 21 18:00:35.215121 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.214570 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 21 18:00:35.215475 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.215456 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 21 18:00:35.215475 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.215467 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 21 18:00:35.215839 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.215781 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 21 18:00:35.215839 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.215790 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 21 18:00:35.215992 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.215847 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 21 18:00:35.215992 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.215881 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-d59xs\"" Jul 21 18:00:35.216241 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.216225 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 21 18:00:35.216241 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.216236 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 21 18:00:35.216372 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.216247 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 21 18:00:35.216626 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.216611 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 21 18:00:35.216704 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.216686 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 21 18:00:35.216772 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.216642 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 21 18:00:35.216877 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.216860 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 21 18:00:35.216969 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.216952 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qmcxf\"" Jul 21 18:00:35.217706 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.217218 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 21 18:00:35.217706 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.217414 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pqxk6\"" Jul 21 18:00:35.218587 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.218540 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6jjkh"] Jul 21 18:00:35.218587 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.218583 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq"] Jul 21 18:00:35.218716 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.218606 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7"] Jul 21 18:00:35.222841 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.222821 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d"] Jul 21 18:00:35.223874 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.223856 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 21 18:00:35.224013 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.223998 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-v5fpd\"" Jul 21 18:00:35.225164 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.225144 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" Jul 21 18:00:35.229303 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.228967 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 21 18:00:35.229303 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.229085 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-c98vg\"" Jul 21 18:00:35.229303 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.229199 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 21 18:00:35.230152 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.230058 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-krmfj"] Jul 21 18:00:35.232464 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.231424 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vvvfz"] Jul 21 18:00:35.234127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.234093 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-cb5pk"] Jul 21 18:00:35.234212 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.234197 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vvvfz" Jul 21 18:00:35.240187 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.240163 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b"] Jul 21 18:00:35.240345 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.240324 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 21 18:00:35.240421 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.240405 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 21 18:00:35.240494 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.240464 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cb5pk" Jul 21 18:00:35.240584 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.240281 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 21 18:00:35.240923 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.240602 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wbjcp\"" Jul 21 18:00:35.242608 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.242589 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-74hdc"] Jul 21 18:00:35.242751 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.242731 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-ztxpr"] Jul 21 18:00:35.242851 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.242686 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" Jul 21 18:00:35.245980 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.245951 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 18:00:35.246160 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246079 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 18:00:35.246310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246293 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-d2hgm\"" Jul 21 18:00:35.246371 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246319 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-bound-sa-token\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.246371 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246352 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2j99\" (UniqueName: \"kubernetes.io/projected/043e03ca-7086-49e3-96b3-b824757c652a-kube-api-access-n2j99\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.246466 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246389 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.246466 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246433 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8586960d-dd9a-45be-89e0-db7ab33d749c-ca-trust-extracted\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.246567 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246462 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8586960d-dd9a-45be-89e0-db7ab33d749c-trusted-ca\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.246567 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246497 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.246567 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246519 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-certificates\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.246567 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246542 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7lfv\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-kube-api-access-g7lfv\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.246760 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246583 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-stats-auth\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.246760 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246638 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-default-certificate\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.246760 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246671 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.246760 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246690 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8586960d-dd9a-45be-89e0-db7ab33d749c-image-registry-private-configuration\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.246760 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.246714 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8586960d-dd9a-45be-89e0-db7ab33d749c-installation-pull-secrets\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.253147 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.253123 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 21 18:00:35.254297 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.254277 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 21 18:00:35.254536 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.254274 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-trsmp\"" Jul 21 18:00:35.254536 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.254312 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 21 18:00:35.254536 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.254316 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 21 18:00:35.257129 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.257104 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf"] Jul 21 18:00:35.261538 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.261477 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b"] Jul 21 18:00:35.262545 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.262523 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-cb5pk"] Jul 21 18:00:35.263408 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.263391 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vvvfz"] Jul 21 18:00:35.281099 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.281075 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d"] Jul 21 18:00:35.348096 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348062 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ncqd\" (UniqueName: \"kubernetes.io/projected/e0d15d58-ba71-482a-9cac-76f87f853a56-kube-api-access-8ncqd\") pod \"network-check-source-7bc5b6bf85-cb5pk\" (UID: \"e0d15d58-ba71-482a-9cac-76f87f853a56\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cb5pk" Jul 21 18:00:35.348096 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348099 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46b0651b-2ded-4eee-b247-cf1404ba7ba3-serving-cert\") pod \"service-ca-operator-5bcbc94c78-tcd2b\" (UID: \"46b0651b-2ded-4eee-b247-cf1404ba7ba3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" Jul 21 18:00:35.348323 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348127 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nqtw\" (UniqueName: \"kubernetes.io/projected/d7074553-ba3e-43c7-8f4c-1037942bb3a9-kube-api-access-2nqtw\") pod \"volume-data-source-validator-78cf4c6f6-6jjkh\" (UID: \"d7074553-ba3e-43c7-8f4c-1037942bb3a9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6jjkh" Jul 21 18:00:35.348323 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348154 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/befe4828-08d1-4685-84ef-766f46b64135-tmp-dir\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:35.348323 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348179 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7bc042d5-1297-4ac5-8237-76c5e2ab7ddd-config\") pod \"console-operator-565b7b569d-74hdc\" (UID: \"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd\") " pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:35.348323 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348200 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7bc042d5-1297-4ac5-8237-76c5e2ab7ddd-serving-cert\") pod \"console-operator-565b7b569d-74hdc\" (UID: \"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd\") " pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:35.348323 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348220 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37231e4d-5ee2-44d9-9482-19501acca31d-cert\") pod \"ingress-canary-vvvfz\" (UID: \"37231e4d-5ee2-44d9-9482-19501acca31d\") " pod="openshift-ingress-canary/ingress-canary-vvvfz" Jul 21 18:00:35.348323 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348267 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gxlqf\" (UID: \"3761b47f-2110-4863-9db7-ffb3a826def2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" Jul 21 18:00:35.348323 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348318 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8586960d-dd9a-45be-89e0-db7ab33d749c-ca-trust-extracted\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.348676 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348360 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:00:35.348676 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348402 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxvxl\" (UniqueName: \"kubernetes.io/projected/3761b47f-2110-4863-9db7-ffb3a826def2-kube-api-access-dxvxl\") pod \"cluster-samples-operator-79b7c55f87-gxlqf\" (UID: \"3761b47f-2110-4863-9db7-ffb3a826def2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" Jul 21 18:00:35.348676 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348434 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qdtc\" (UniqueName: \"kubernetes.io/projected/80cad807-9199-42c6-8b46-8d6887025525-kube-api-access-6qdtc\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.348676 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348461 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80cad807-9199-42c6-8b46-8d6887025525-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.348676 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348482 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d4717f23-7fe1-4c06-b411-0bb095e21361-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:00:35.348676 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348540 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-certificates\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.348676 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348574 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7lfv\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-kube-api-access-g7lfv\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.348676 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348603 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7bc042d5-1297-4ac5-8237-76c5e2ab7ddd-trusted-ca\") pod \"console-operator-565b7b569d-74hdc\" (UID: \"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd\") " pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:35.348676 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348630 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf634bfc-df89-4e85-874a-33bd28e7eb68-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-llddq\" (UID: \"cf634bfc-df89-4e85-874a-33bd28e7eb68\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" Jul 21 18:00:35.349150 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348775 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.349150 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348825 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.349150 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348855 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-zvk8d\" (UID: \"8d00aa82-0b34-4f8b-a7a1-3c4e3458833f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" Jul 21 18:00:35.349150 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348826 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8586960d-dd9a-45be-89e0-db7ab33d749c-ca-trust-extracted\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.349150 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348888 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wblt8\" (UniqueName: \"kubernetes.io/projected/cf634bfc-df89-4e85-874a-33bd28e7eb68-kube-api-access-wblt8\") pod \"kube-storage-version-migrator-operator-7658d9b749-llddq\" (UID: \"cf634bfc-df89-4e85-874a-33bd28e7eb68\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" Jul 21 18:00:35.349150 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.348915 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 18:00:35.349150 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348919 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-zvk8d\" (UID: \"8d00aa82-0b34-4f8b-a7a1-3c4e3458833f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" Jul 21 18:00:35.349150 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.348956 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6774fd47f-flh6l: secret "image-registry-tls" not found Jul 21 18:00:35.349150 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.348998 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:35.349150 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.349001 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 18:00:35.349150 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.349015 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls podName:8586960d-dd9a-45be-89e0-db7ab33d749c nodeName:}" failed. No retries permitted until 2026-07-21 18:00:35.848995026 +0000 UTC m=+37.117306457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls") pod "image-registry-6774fd47f-flh6l" (UID: "8586960d-dd9a-45be-89e0-db7ab33d749c") : secret "image-registry-tls" not found Jul 21 18:00:35.349150 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349139 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-certificates\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.349729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349212 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxhmk\" (UniqueName: \"kubernetes.io/projected/d4717f23-7fe1-4c06-b411-0bb095e21361-kube-api-access-jxhmk\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:00:35.349729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349245 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.349729 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.349281 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs podName:043e03ca-7086-49e3-96b3-b824757c652a nodeName:}" failed. No retries permitted until 2026-07-21 18:00:35.849260123 +0000 UTC m=+37.117571545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs") pod "router-default-799cfd6768-8vvdp" (UID: "043e03ca-7086-49e3-96b3-b824757c652a") : secret "router-metrics-certs-default" not found Jul 21 18:00:35.349729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349336 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80cad807-9199-42c6-8b46-8d6887025525-tmp\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.349729 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.349378 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle podName:043e03ca-7086-49e3-96b3-b824757c652a nodeName:}" failed. No retries permitted until 2026-07-21 18:00:35.849367058 +0000 UTC m=+37.117678500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle") pod "router-default-799cfd6768-8vvdp" (UID: "043e03ca-7086-49e3-96b3-b824757c652a") : configmap references non-existent config key: service-ca.crt Jul 21 18:00:35.349729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349397 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8586960d-dd9a-45be-89e0-db7ab33d749c-trusted-ca\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.349729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349421 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/80cad807-9199-42c6-8b46-8d6887025525-serving-cert\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.349729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349456 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46b0651b-2ded-4eee-b247-cf1404ba7ba3-config\") pod \"service-ca-operator-5bcbc94c78-tcd2b\" (UID: \"46b0651b-2ded-4eee-b247-cf1404ba7ba3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" Jul 21 18:00:35.349729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349493 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-stats-auth\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.349729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349520 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6df65\" (UniqueName: \"kubernetes.io/projected/befe4828-08d1-4685-84ef-766f46b64135-kube-api-access-6df65\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:35.349729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349553 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-default-certificate\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.349729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349582 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/80cad807-9199-42c6-8b46-8d6887025525-snapshots\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.350313 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349613 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8586960d-dd9a-45be-89e0-db7ab33d749c-image-registry-private-configuration\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.350313 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349645 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8586960d-dd9a-45be-89e0-db7ab33d749c-installation-pull-secrets\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.350313 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349691 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/befe4828-08d1-4685-84ef-766f46b64135-config-volume\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:35.350313 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349715 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8qwq\" (UniqueName: \"kubernetes.io/projected/37231e4d-5ee2-44d9-9482-19501acca31d-kube-api-access-d8qwq\") pod \"ingress-canary-vvvfz\" (UID: \"37231e4d-5ee2-44d9-9482-19501acca31d\") " pod="openshift-ingress-canary/ingress-canary-vvvfz" Jul 21 18:00:35.350313 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349743 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xws7\" (UniqueName: \"kubernetes.io/projected/7bc042d5-1297-4ac5-8237-76c5e2ab7ddd-kube-api-access-6xws7\") pod \"console-operator-565b7b569d-74hdc\" (UID: \"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd\") " pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:35.350313 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349760 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf634bfc-df89-4e85-874a-33bd28e7eb68-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-llddq\" (UID: \"cf634bfc-df89-4e85-874a-33bd28e7eb68\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" Jul 21 18:00:35.350313 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349786 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-bound-sa-token\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.350313 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349805 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6jhd\" (UniqueName: \"kubernetes.io/projected/46b0651b-2ded-4eee-b247-cf1404ba7ba3-kube-api-access-w6jhd\") pod \"service-ca-operator-5bcbc94c78-tcd2b\" (UID: \"46b0651b-2ded-4eee-b247-cf1404ba7ba3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" Jul 21 18:00:35.350313 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349820 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80cad807-9199-42c6-8b46-8d6887025525-service-ca-bundle\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.350313 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.349841 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2j99\" (UniqueName: \"kubernetes.io/projected/043e03ca-7086-49e3-96b3-b824757c652a-kube-api-access-n2j99\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.350313 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.350207 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8586960d-dd9a-45be-89e0-db7ab33d749c-trusted-ca\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.354467 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.354442 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8586960d-dd9a-45be-89e0-db7ab33d749c-image-registry-private-configuration\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.354467 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.354453 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-default-certificate\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.354611 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.354466 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-stats-auth\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.354611 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.354443 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8586960d-dd9a-45be-89e0-db7ab33d749c-installation-pull-secrets\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.369154 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.369135 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2j99\" (UniqueName: \"kubernetes.io/projected/043e03ca-7086-49e3-96b3-b824757c652a-kube-api-access-n2j99\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.372487 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.372460 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7lfv\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-kube-api-access-g7lfv\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.376429 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.376402 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-bound-sa-token\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.412246 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.412206 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:35.450930 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.450896 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:35.450930 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.450949 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxhmk\" (UniqueName: \"kubernetes.io/projected/d4717f23-7fe1-4c06-b411-0bb095e21361-kube-api-access-jxhmk\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:00:35.451170 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.451063 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 18:00:35.451170 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451070 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80cad807-9199-42c6-8b46-8d6887025525-tmp\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.451170 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451106 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/80cad807-9199-42c6-8b46-8d6887025525-serving-cert\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.451170 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.451127 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls podName:befe4828-08d1-4685-84ef-766f46b64135 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:35.951107567 +0000 UTC m=+37.219419015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls") pod "dns-default-krmfj" (UID: "befe4828-08d1-4685-84ef-766f46b64135") : secret "dns-default-metrics-tls" not found Jul 21 18:00:35.451170 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451159 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46b0651b-2ded-4eee-b247-cf1404ba7ba3-config\") pod \"service-ca-operator-5bcbc94c78-tcd2b\" (UID: \"46b0651b-2ded-4eee-b247-cf1404ba7ba3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" Jul 21 18:00:35.451427 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451193 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6df65\" (UniqueName: \"kubernetes.io/projected/befe4828-08d1-4685-84ef-766f46b64135-kube-api-access-6df65\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:35.451427 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451220 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/80cad807-9199-42c6-8b46-8d6887025525-snapshots\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.451427 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451254 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/befe4828-08d1-4685-84ef-766f46b64135-config-volume\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:35.451427 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451278 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8qwq\" (UniqueName: \"kubernetes.io/projected/37231e4d-5ee2-44d9-9482-19501acca31d-kube-api-access-d8qwq\") pod \"ingress-canary-vvvfz\" (UID: \"37231e4d-5ee2-44d9-9482-19501acca31d\") " pod="openshift-ingress-canary/ingress-canary-vvvfz" Jul 21 18:00:35.451427 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451318 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xws7\" (UniqueName: \"kubernetes.io/projected/7bc042d5-1297-4ac5-8237-76c5e2ab7ddd-kube-api-access-6xws7\") pod \"console-operator-565b7b569d-74hdc\" (UID: \"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd\") " pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:35.451427 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451347 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf634bfc-df89-4e85-874a-33bd28e7eb68-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-llddq\" (UID: \"cf634bfc-df89-4e85-874a-33bd28e7eb68\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" Jul 21 18:00:35.451427 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451376 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6jhd\" (UniqueName: \"kubernetes.io/projected/46b0651b-2ded-4eee-b247-cf1404ba7ba3-kube-api-access-w6jhd\") pod \"service-ca-operator-5bcbc94c78-tcd2b\" (UID: \"46b0651b-2ded-4eee-b247-cf1404ba7ba3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" Jul 21 18:00:35.451427 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451401 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80cad807-9199-42c6-8b46-8d6887025525-service-ca-bundle\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.451814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451443 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ncqd\" (UniqueName: \"kubernetes.io/projected/e0d15d58-ba71-482a-9cac-76f87f853a56-kube-api-access-8ncqd\") pod \"network-check-source-7bc5b6bf85-cb5pk\" (UID: \"e0d15d58-ba71-482a-9cac-76f87f853a56\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cb5pk" Jul 21 18:00:35.451814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451468 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46b0651b-2ded-4eee-b247-cf1404ba7ba3-serving-cert\") pod \"service-ca-operator-5bcbc94c78-tcd2b\" (UID: \"46b0651b-2ded-4eee-b247-cf1404ba7ba3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" Jul 21 18:00:35.451814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451498 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nqtw\" (UniqueName: \"kubernetes.io/projected/d7074553-ba3e-43c7-8f4c-1037942bb3a9-kube-api-access-2nqtw\") pod \"volume-data-source-validator-78cf4c6f6-6jjkh\" (UID: \"d7074553-ba3e-43c7-8f4c-1037942bb3a9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6jjkh" Jul 21 18:00:35.451814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451526 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/befe4828-08d1-4685-84ef-766f46b64135-tmp-dir\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:35.451814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451551 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7bc042d5-1297-4ac5-8237-76c5e2ab7ddd-config\") pod \"console-operator-565b7b569d-74hdc\" (UID: \"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd\") " pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:35.451814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451561 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80cad807-9199-42c6-8b46-8d6887025525-tmp\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.451814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451577 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7bc042d5-1297-4ac5-8237-76c5e2ab7ddd-serving-cert\") pod \"console-operator-565b7b569d-74hdc\" (UID: \"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd\") " pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:35.451814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451614 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37231e4d-5ee2-44d9-9482-19501acca31d-cert\") pod \"ingress-canary-vvvfz\" (UID: \"37231e4d-5ee2-44d9-9482-19501acca31d\") " pod="openshift-ingress-canary/ingress-canary-vvvfz" Jul 21 18:00:35.451814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451643 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gxlqf\" (UID: \"3761b47f-2110-4863-9db7-ffb3a826def2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" Jul 21 18:00:35.451814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451675 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:00:35.451814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451702 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxvxl\" (UniqueName: \"kubernetes.io/projected/3761b47f-2110-4863-9db7-ffb3a826def2-kube-api-access-dxvxl\") pod \"cluster-samples-operator-79b7c55f87-gxlqf\" (UID: \"3761b47f-2110-4863-9db7-ffb3a826def2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" Jul 21 18:00:35.451814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451729 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qdtc\" (UniqueName: \"kubernetes.io/projected/80cad807-9199-42c6-8b46-8d6887025525-kube-api-access-6qdtc\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.451814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451753 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46b0651b-2ded-4eee-b247-cf1404ba7ba3-config\") pod \"service-ca-operator-5bcbc94c78-tcd2b\" (UID: \"46b0651b-2ded-4eee-b247-cf1404ba7ba3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" Jul 21 18:00:35.452435 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451760 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80cad807-9199-42c6-8b46-8d6887025525-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.452435 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451787 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d4717f23-7fe1-4c06-b411-0bb095e21361-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:00:35.452435 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451821 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7bc042d5-1297-4ac5-8237-76c5e2ab7ddd-trusted-ca\") pod \"console-operator-565b7b569d-74hdc\" (UID: \"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd\") " pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:35.452435 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451846 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf634bfc-df89-4e85-874a-33bd28e7eb68-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-llddq\" (UID: \"cf634bfc-df89-4e85-874a-33bd28e7eb68\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" Jul 21 18:00:35.452435 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451918 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-zvk8d\" (UID: \"8d00aa82-0b34-4f8b-a7a1-3c4e3458833f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" Jul 21 18:00:35.452435 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.451970 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wblt8\" (UniqueName: \"kubernetes.io/projected/cf634bfc-df89-4e85-874a-33bd28e7eb68-kube-api-access-wblt8\") pod \"kube-storage-version-migrator-operator-7658d9b749-llddq\" (UID: \"cf634bfc-df89-4e85-874a-33bd28e7eb68\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" Jul 21 18:00:35.452435 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.452213 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-zvk8d\" (UID: \"8d00aa82-0b34-4f8b-a7a1-3c4e3458833f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" Jul 21 18:00:35.452435 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.452258 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 18:00:35.452435 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.452322 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls podName:3761b47f-2110-4863-9db7-ffb3a826def2 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:35.952296431 +0000 UTC m=+37.220607864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gxlqf" (UID: "3761b47f-2110-4863-9db7-ffb3a826def2") : secret "samples-operator-tls" not found Jul 21 18:00:35.452435 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.452388 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 18:00:35.452435 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.452427 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls podName:d4717f23-7fe1-4c06-b411-0bb095e21361 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:35.952415586 +0000 UTC m=+37.220727020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tvbh7" (UID: "d4717f23-7fe1-4c06-b411-0bb095e21361") : secret "cluster-monitoring-operator-tls" not found Jul 21 18:00:35.452971 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.452496 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/80cad807-9199-42c6-8b46-8d6887025525-snapshots\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.453928 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.453036 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-zvk8d\" (UID: \"8d00aa82-0b34-4f8b-a7a1-3c4e3458833f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" Jul 21 18:00:35.453928 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.453107 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80cad807-9199-42c6-8b46-8d6887025525-service-ca-bundle\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.453928 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.453239 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80cad807-9199-42c6-8b46-8d6887025525-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.453928 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.453241 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf634bfc-df89-4e85-874a-33bd28e7eb68-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-llddq\" (UID: \"cf634bfc-df89-4e85-874a-33bd28e7eb68\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" Jul 21 18:00:35.453928 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.453343 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 18:00:35.453928 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.453437 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert podName:8d00aa82-0b34-4f8b-a7a1-3c4e3458833f nodeName:}" failed. No retries permitted until 2026-07-21 18:00:35.953419884 +0000 UTC m=+37.221731319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-zvk8d" (UID: "8d00aa82-0b34-4f8b-a7a1-3c4e3458833f") : secret "networking-console-plugin-cert" not found Jul 21 18:00:35.453928 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.453467 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7bc042d5-1297-4ac5-8237-76c5e2ab7ddd-trusted-ca\") pod \"console-operator-565b7b569d-74hdc\" (UID: \"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd\") " pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:35.453928 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.453648 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/befe4828-08d1-4685-84ef-766f46b64135-config-volume\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:35.453928 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.453721 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/befe4828-08d1-4685-84ef-766f46b64135-tmp-dir\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:35.454449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.454016 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d4717f23-7fe1-4c06-b411-0bb095e21361-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:00:35.454449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.454251 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7bc042d5-1297-4ac5-8237-76c5e2ab7ddd-config\") pod \"console-operator-565b7b569d-74hdc\" (UID: \"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd\") " pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:35.454449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.454286 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/80cad807-9199-42c6-8b46-8d6887025525-serving-cert\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.454694 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.454675 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7bc042d5-1297-4ac5-8237-76c5e2ab7ddd-serving-cert\") pod \"console-operator-565b7b569d-74hdc\" (UID: \"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd\") " pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:35.456059 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.456041 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46b0651b-2ded-4eee-b247-cf1404ba7ba3-serving-cert\") pod \"service-ca-operator-5bcbc94c78-tcd2b\" (UID: \"46b0651b-2ded-4eee-b247-cf1404ba7ba3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" Jul 21 18:00:35.456261 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.456221 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf634bfc-df89-4e85-874a-33bd28e7eb68-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-llddq\" (UID: \"cf634bfc-df89-4e85-874a-33bd28e7eb68\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" Jul 21 18:00:35.456417 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.456399 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37231e4d-5ee2-44d9-9482-19501acca31d-cert\") pod \"ingress-canary-vvvfz\" (UID: \"37231e4d-5ee2-44d9-9482-19501acca31d\") " pod="openshift-ingress-canary/ingress-canary-vvvfz" Jul 21 18:00:35.490823 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.490778 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wblt8\" (UniqueName: \"kubernetes.io/projected/cf634bfc-df89-4e85-874a-33bd28e7eb68-kube-api-access-wblt8\") pod \"kube-storage-version-migrator-operator-7658d9b749-llddq\" (UID: \"cf634bfc-df89-4e85-874a-33bd28e7eb68\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" Jul 21 18:00:35.492099 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.491994 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qdtc\" (UniqueName: \"kubernetes.io/projected/80cad807-9199-42c6-8b46-8d6887025525-kube-api-access-6qdtc\") pod \"insights-operator-555d9d7885-ztxpr\" (UID: \"80cad807-9199-42c6-8b46-8d6887025525\") " pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.492099 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.492053 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nqtw\" (UniqueName: \"kubernetes.io/projected/d7074553-ba3e-43c7-8f4c-1037942bb3a9-kube-api-access-2nqtw\") pod \"volume-data-source-validator-78cf4c6f6-6jjkh\" (UID: \"d7074553-ba3e-43c7-8f4c-1037942bb3a9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6jjkh" Jul 21 18:00:35.492099 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.492088 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxhmk\" (UniqueName: \"kubernetes.io/projected/d4717f23-7fe1-4c06-b411-0bb095e21361-kube-api-access-jxhmk\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:00:35.492317 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.492121 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8qwq\" (UniqueName: \"kubernetes.io/projected/37231e4d-5ee2-44d9-9482-19501acca31d-kube-api-access-d8qwq\") pod \"ingress-canary-vvvfz\" (UID: \"37231e4d-5ee2-44d9-9482-19501acca31d\") " pod="openshift-ingress-canary/ingress-canary-vvvfz" Jul 21 18:00:35.493003 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.492960 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6jhd\" (UniqueName: \"kubernetes.io/projected/46b0651b-2ded-4eee-b247-cf1404ba7ba3-kube-api-access-w6jhd\") pod \"service-ca-operator-5bcbc94c78-tcd2b\" (UID: \"46b0651b-2ded-4eee-b247-cf1404ba7ba3\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" Jul 21 18:00:35.493119 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.493068 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xws7\" (UniqueName: \"kubernetes.io/projected/7bc042d5-1297-4ac5-8237-76c5e2ab7ddd-kube-api-access-6xws7\") pod \"console-operator-565b7b569d-74hdc\" (UID: \"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd\") " pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:35.499243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.499221 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-ztxpr" Jul 21 18:00:35.502762 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.502737 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6df65\" (UniqueName: \"kubernetes.io/projected/befe4828-08d1-4685-84ef-766f46b64135-kube-api-access-6df65\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:35.503131 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.503106 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ncqd\" (UniqueName: \"kubernetes.io/projected/e0d15d58-ba71-482a-9cac-76f87f853a56-kube-api-access-8ncqd\") pod \"network-check-source-7bc5b6bf85-cb5pk\" (UID: \"e0d15d58-ba71-482a-9cac-76f87f853a56\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cb5pk" Jul 21 18:00:35.503210 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.503138 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxvxl\" (UniqueName: \"kubernetes.io/projected/3761b47f-2110-4863-9db7-ffb3a826def2-kube-api-access-dxvxl\") pod \"cluster-samples-operator-79b7c55f87-gxlqf\" (UID: \"3761b47f-2110-4863-9db7-ffb3a826def2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" Jul 21 18:00:35.517608 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.517584 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6jjkh" Jul 21 18:00:35.534342 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.534292 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" Jul 21 18:00:35.553581 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.553545 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:35.571249 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.571222 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vvvfz" Jul 21 18:00:35.577592 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.577568 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cb5pk" Jul 21 18:00:35.583267 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.583242 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" Jul 21 18:00:35.590901 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.590872 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:35.660478 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.660002 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:35.856285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.856248 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:35.856479 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.856308 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.856479 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.856355 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:35.856479 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.856412 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 18:00:35.856479 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.856436 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6774fd47f-flh6l: secret "image-registry-tls" not found Jul 21 18:00:35.856479 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.856469 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 18:00:35.856720 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.856501 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls podName:8586960d-dd9a-45be-89e0-db7ab33d749c nodeName:}" failed. No retries permitted until 2026-07-21 18:00:36.856481255 +0000 UTC m=+38.124792678 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls") pod "image-registry-6774fd47f-flh6l" (UID: "8586960d-dd9a-45be-89e0-db7ab33d749c") : secret "image-registry-tls" not found Jul 21 18:00:35.856720 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.856528 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs podName:043e03ca-7086-49e3-96b3-b824757c652a nodeName:}" failed. No retries permitted until 2026-07-21 18:00:36.856510084 +0000 UTC m=+38.124821507 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs") pod "router-default-799cfd6768-8vvdp" (UID: "043e03ca-7086-49e3-96b3-b824757c652a") : secret "router-metrics-certs-default" not found Jul 21 18:00:35.856720 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.856598 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle podName:043e03ca-7086-49e3-96b3-b824757c652a nodeName:}" failed. No retries permitted until 2026-07-21 18:00:36.856585416 +0000 UTC m=+38.124896844 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle") pod "router-default-799cfd6768-8vvdp" (UID: "043e03ca-7086-49e3-96b3-b824757c652a") : configmap references non-existent config key: service-ca.crt Jul 21 18:00:35.957665 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.957571 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gxlqf\" (UID: \"3761b47f-2110-4863-9db7-ffb3a826def2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" Jul 21 18:00:35.957665 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.957617 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:00:35.957892 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.957688 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-zvk8d\" (UID: \"8d00aa82-0b34-4f8b-a7a1-3c4e3458833f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" Jul 21 18:00:35.957892 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.957736 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 18:00:35.957892 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.957766 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 18:00:35.957892 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.957804 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls podName:3761b47f-2110-4863-9db7-ffb3a826def2 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:36.957784724 +0000 UTC m=+38.226096161 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gxlqf" (UID: "3761b47f-2110-4863-9db7-ffb3a826def2") : secret "samples-operator-tls" not found Jul 21 18:00:35.957892 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.957817 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 18:00:35.957892 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.957822 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert podName:8d00aa82-0b34-4f8b-a7a1-3c4e3458833f nodeName:}" failed. No retries permitted until 2026-07-21 18:00:36.95781312 +0000 UTC m=+38.226124540 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-zvk8d" (UID: "8d00aa82-0b34-4f8b-a7a1-3c4e3458833f") : secret "networking-console-plugin-cert" not found Jul 21 18:00:35.958188 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:35.957916 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:35.958188 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.957996 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 18:00:35.958188 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.958013 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls podName:d4717f23-7fe1-4c06-b411-0bb095e21361 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:36.957998062 +0000 UTC m=+38.226309486 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tvbh7" (UID: "d4717f23-7fe1-4c06-b411-0bb095e21361") : secret "cluster-monitoring-operator-tls" not found Jul 21 18:00:35.958188 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:35.958051 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls podName:befe4828-08d1-4685-84ef-766f46b64135 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:36.958032791 +0000 UTC m=+38.226344212 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls") pod "dns-default-krmfj" (UID: "befe4828-08d1-4685-84ef-766f46b64135") : secret "dns-default-metrics-tls" not found Jul 21 18:00:36.213379 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.213291 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:00:36.213530 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.213294 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:00:36.213530 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.213293 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:00:36.216034 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.216009 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 18:00:36.216619 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.216108 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vznt9\"" Jul 21 18:00:36.217196 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.217113 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jg5zx\"" Jul 21 18:00:36.217196 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.217155 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 18:00:36.339948 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.339907 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:36.421172 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.421133 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:36.871481 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.870485 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:36.871481 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.870863 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:36.871481 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.870908 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:36.871481 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:36.871199 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle podName:043e03ca-7086-49e3-96b3-b824757c652a nodeName:}" failed. No retries permitted until 2026-07-21 18:00:38.8711771 +0000 UTC m=+40.139488523 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle") pod "router-default-799cfd6768-8vvdp" (UID: "043e03ca-7086-49e3-96b3-b824757c652a") : configmap references non-existent config key: service-ca.crt Jul 21 18:00:36.876121 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:36.871884 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 18:00:36.876121 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:36.871905 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6774fd47f-flh6l: secret "image-registry-tls" not found Jul 21 18:00:36.876121 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:36.871971 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls podName:8586960d-dd9a-45be-89e0-db7ab33d749c nodeName:}" failed. No retries permitted until 2026-07-21 18:00:38.871954592 +0000 UTC m=+40.140266019 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls") pod "image-registry-6774fd47f-flh6l" (UID: "8586960d-dd9a-45be-89e0-db7ab33d749c") : secret "image-registry-tls" not found Jul 21 18:00:36.876121 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:36.872173 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 18:00:36.876121 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:36.872216 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs podName:043e03ca-7086-49e3-96b3-b824757c652a nodeName:}" failed. No retries permitted until 2026-07-21 18:00:38.872200673 +0000 UTC m=+40.140512095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs") pod "router-default-799cfd6768-8vvdp" (UID: "043e03ca-7086-49e3-96b3-b824757c652a") : secret "router-metrics-certs-default" not found Jul 21 18:00:36.987630 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.985967 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gxlqf\" (UID: \"3761b47f-2110-4863-9db7-ffb3a826def2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" Jul 21 18:00:36.987630 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.986030 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:00:36.987630 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.986115 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-zvk8d\" (UID: \"8d00aa82-0b34-4f8b-a7a1-3c4e3458833f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" Jul 21 18:00:36.987630 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:36.986135 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 18:00:36.987630 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:36.986162 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:36.987630 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:36.986212 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls podName:3761b47f-2110-4863-9db7-ffb3a826def2 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:38.986190656 +0000 UTC m=+40.254502095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gxlqf" (UID: "3761b47f-2110-4863-9db7-ffb3a826def2") : secret "samples-operator-tls" not found Jul 21 18:00:36.987630 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:36.986284 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 18:00:36.987630 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:36.986336 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls podName:befe4828-08d1-4685-84ef-766f46b64135 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:38.986321222 +0000 UTC m=+40.254632660 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls") pod "dns-default-krmfj" (UID: "befe4828-08d1-4685-84ef-766f46b64135") : secret "dns-default-metrics-tls" not found Jul 21 18:00:36.987630 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:36.986393 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 18:00:36.987630 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:36.986425 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls podName:d4717f23-7fe1-4c06-b411-0bb095e21361 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:38.986414194 +0000 UTC m=+40.254725634 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tvbh7" (UID: "d4717f23-7fe1-4c06-b411-0bb095e21361") : secret "cluster-monitoring-operator-tls" not found Jul 21 18:00:36.987630 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:36.986483 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 18:00:36.987630 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:36.986515 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert podName:8d00aa82-0b34-4f8b-a7a1-3c4e3458833f nodeName:}" failed. No retries permitted until 2026-07-21 18:00:38.986503532 +0000 UTC m=+40.254814989 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-zvk8d" (UID: "8d00aa82-0b34-4f8b-a7a1-3c4e3458833f") : secret "networking-console-plugin-cert" not found Jul 21 18:00:37.058530 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.058491 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq"] Jul 21 18:00:37.064769 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.064636 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b"] Jul 21 18:00:37.065974 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.065954 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-ztxpr"] Jul 21 18:00:37.067145 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.067127 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6jjkh"] Jul 21 18:00:37.077993 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.077870 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-cb5pk"] Jul 21 18:00:37.079460 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.079299 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-74hdc"] Jul 21 18:00:37.080488 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.080468 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vvvfz"] Jul 21 18:00:37.083568 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:37.083541 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7074553_ba3e_43c7_8f4c_1037942bb3a9.slice/crio-bc6516d97809a3ef811ce1a7e7510abf9bc164bd342defb1f8ab6a4ad51ab5ac WatchSource:0}: Error finding container bc6516d97809a3ef811ce1a7e7510abf9bc164bd342defb1f8ab6a4ad51ab5ac: Status 404 returned error can't find the container with id bc6516d97809a3ef811ce1a7e7510abf9bc164bd342defb1f8ab6a4ad51ab5ac Jul 21 18:00:37.084715 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:37.084672 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37231e4d_5ee2_44d9_9482_19501acca31d.slice/crio-2f141019c8865f8b7da303b5010a96a2b36109f6a9693b87dc1511f36527f22a WatchSource:0}: Error finding container 2f141019c8865f8b7da303b5010a96a2b36109f6a9693b87dc1511f36527f22a: Status 404 returned error can't find the container with id 2f141019c8865f8b7da303b5010a96a2b36109f6a9693b87dc1511f36527f22a Jul 21 18:00:37.085770 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:37.085750 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bc042d5_1297_4ac5_8237_76c5e2ab7ddd.slice/crio-c08937c2c521d64ba73d6b56d2be903bbd79057276942aacfd9f37046c508269 WatchSource:0}: Error finding container c08937c2c521d64ba73d6b56d2be903bbd79057276942aacfd9f37046c508269: Status 404 returned error can't find the container with id c08937c2c521d64ba73d6b56d2be903bbd79057276942aacfd9f37046c508269 Jul 21 18:00:37.086980 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:37.086716 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d15d58_ba71_482a_9cac_76f87f853a56.slice/crio-e5546892204c46df108deb612d0757b67f662c6b044f8d41ddd7db97c9891673 WatchSource:0}: Error finding container e5546892204c46df108deb612d0757b67f662c6b044f8d41ddd7db97c9891673: Status 404 returned error can't find the container with id e5546892204c46df108deb612d0757b67f662c6b044f8d41ddd7db97c9891673 Jul 21 18:00:37.089288 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:37.088350 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46b0651b_2ded_4eee_b247_cf1404ba7ba3.slice/crio-abe58e3e43a99dd9147c331926d971f6885923fba42a4d91922d88945dfd549b WatchSource:0}: Error finding container abe58e3e43a99dd9147c331926d971f6885923fba42a4d91922d88945dfd549b: Status 404 returned error can't find the container with id abe58e3e43a99dd9147c331926d971f6885923fba42a4d91922d88945dfd549b Jul 21 18:00:37.090091 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:37.090056 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80cad807_9199_42c6_8b46_8d6887025525.slice/crio-eabd1fa5edd1704e815c05db66f000d16f351fc83003d9300d1c3154483788b3 WatchSource:0}: Error finding container eabd1fa5edd1704e815c05db66f000d16f351fc83003d9300d1c3154483788b3: Status 404 returned error can't find the container with id eabd1fa5edd1704e815c05db66f000d16f351fc83003d9300d1c3154483788b3 Jul 21 18:00:37.090382 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:37.090359 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf634bfc_df89_4e85_874a_33bd28e7eb68.slice/crio-4e683721e9fa3a991db11d21eb9300d517e81a96e0c0f7f10c95c1735cd3e10b WatchSource:0}: Error finding container 4e683721e9fa3a991db11d21eb9300d517e81a96e0c0f7f10c95c1735cd3e10b: Status 404 returned error can't find the container with id 4e683721e9fa3a991db11d21eb9300d517e81a96e0c0f7f10c95c1735cd3e10b Jul 21 18:00:37.384404 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.384368 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cb5pk" event={"ID":"e0d15d58-ba71-482a-9cac-76f87f853a56","Type":"ContainerStarted","Data":"e5546892204c46df108deb612d0757b67f662c6b044f8d41ddd7db97c9891673"} Jul 21 18:00:37.385362 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.385337 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" event={"ID":"cf634bfc-df89-4e85-874a-33bd28e7eb68","Type":"ContainerStarted","Data":"4e683721e9fa3a991db11d21eb9300d517e81a96e0c0f7f10c95c1735cd3e10b"} Jul 21 18:00:37.386324 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.386302 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" event={"ID":"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd","Type":"ContainerStarted","Data":"c08937c2c521d64ba73d6b56d2be903bbd79057276942aacfd9f37046c508269"} Jul 21 18:00:37.387303 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.387283 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" event={"ID":"46b0651b-2ded-4eee-b247-cf1404ba7ba3","Type":"ContainerStarted","Data":"abe58e3e43a99dd9147c331926d971f6885923fba42a4d91922d88945dfd549b"} Jul 21 18:00:37.388315 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.388296 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ztxpr" event={"ID":"80cad807-9199-42c6-8b46-8d6887025525","Type":"ContainerStarted","Data":"eabd1fa5edd1704e815c05db66f000d16f351fc83003d9300d1c3154483788b3"} Jul 21 18:00:37.389338 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.389321 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vvvfz" event={"ID":"37231e4d-5ee2-44d9-9482-19501acca31d","Type":"ContainerStarted","Data":"2f141019c8865f8b7da303b5010a96a2b36109f6a9693b87dc1511f36527f22a"} Jul 21 18:00:37.390376 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.390357 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6jjkh" event={"ID":"d7074553-ba3e-43c7-8f4c-1037942bb3a9","Type":"ContainerStarted","Data":"bc6516d97809a3ef811ce1a7e7510abf9bc164bd342defb1f8ab6a4ad51ab5ac"} Jul 21 18:00:37.393233 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.393211 2568 generic.go:356] "Generic (PLEG): container finished" podID="3257df56-1fd0-4f11-a9fa-b68d6daa1a1b" containerID="7857780aba71796022300976c43ba0a618529f64265cb96230edcbda06e74c57" exitCode=0 Jul 21 18:00:37.393309 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:37.393243 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8fqf" event={"ID":"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b","Type":"ContainerDied","Data":"7857780aba71796022300976c43ba0a618529f64265cb96230edcbda06e74c57"} Jul 21 18:00:38.408542 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:38.408487 2568 generic.go:356] "Generic (PLEG): container finished" podID="3257df56-1fd0-4f11-a9fa-b68d6daa1a1b" containerID="b7e28a664d65d4fdcab73481759b84957a23bc70428e7715063a5c0ee3c40dd1" exitCode=0 Jul 21 18:00:38.409066 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:38.408572 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8fqf" event={"ID":"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b","Type":"ContainerDied","Data":"b7e28a664d65d4fdcab73481759b84957a23bc70428e7715063a5c0ee3c40dd1"} Jul 21 18:00:38.907058 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:38.906758 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:38.907297 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:38.907095 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:38.907359 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:38.907300 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 18:00:38.907359 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:38.907317 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6774fd47f-flh6l: secret "image-registry-tls" not found Jul 21 18:00:38.907729 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:38.907694 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 18:00:38.907802 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:38.907761 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls podName:8586960d-dd9a-45be-89e0-db7ab33d749c nodeName:}" failed. No retries permitted until 2026-07-21 18:00:42.907736291 +0000 UTC m=+44.176047715 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls") pod "image-registry-6774fd47f-flh6l" (UID: "8586960d-dd9a-45be-89e0-db7ab33d749c") : secret "image-registry-tls" not found Jul 21 18:00:38.907802 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:38.907798 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs podName:043e03ca-7086-49e3-96b3-b824757c652a nodeName:}" failed. No retries permitted until 2026-07-21 18:00:42.907787298 +0000 UTC m=+44.176098723 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs") pod "router-default-799cfd6768-8vvdp" (UID: "043e03ca-7086-49e3-96b3-b824757c652a") : secret "router-metrics-certs-default" not found Jul 21 18:00:38.907910 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:38.907849 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:38.907986 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:38.907949 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle podName:043e03ca-7086-49e3-96b3-b824757c652a nodeName:}" failed. No retries permitted until 2026-07-21 18:00:42.907913288 +0000 UTC m=+44.176224731 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle") pod "router-default-799cfd6768-8vvdp" (UID: "043e03ca-7086-49e3-96b3-b824757c652a") : configmap references non-existent config key: service-ca.crt Jul 21 18:00:39.008830 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:39.008640 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gxlqf\" (UID: \"3761b47f-2110-4863-9db7-ffb3a826def2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" Jul 21 18:00:39.008830 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:39.008699 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:00:39.008830 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:39.008782 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-zvk8d\" (UID: \"8d00aa82-0b34-4f8b-a7a1-3c4e3458833f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" Jul 21 18:00:39.009167 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:39.008970 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 18:00:39.009167 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:39.009038 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls podName:3761b47f-2110-4863-9db7-ffb3a826def2 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:43.009018003 +0000 UTC m=+44.277329428 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gxlqf" (UID: "3761b47f-2110-4863-9db7-ffb3a826def2") : secret "samples-operator-tls" not found Jul 21 18:00:39.009320 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:39.009267 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:39.009517 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:39.009382 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 18:00:39.009517 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:39.009446 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 18:00:39.009517 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:39.009460 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert podName:8d00aa82-0b34-4f8b-a7a1-3c4e3458833f nodeName:}" failed. No retries permitted until 2026-07-21 18:00:43.009440351 +0000 UTC m=+44.277751796 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-zvk8d" (UID: "8d00aa82-0b34-4f8b-a7a1-3c4e3458833f") : secret "networking-console-plugin-cert" not found Jul 21 18:00:39.009517 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:39.009494 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls podName:befe4828-08d1-4685-84ef-766f46b64135 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:43.00948062 +0000 UTC m=+44.277792063 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls") pod "dns-default-krmfj" (UID: "befe4828-08d1-4685-84ef-766f46b64135") : secret "dns-default-metrics-tls" not found Jul 21 18:00:39.009517 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:39.009451 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 18:00:39.009913 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:39.009537 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls podName:d4717f23-7fe1-4c06-b411-0bb095e21361 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:43.009527224 +0000 UTC m=+44.277838649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tvbh7" (UID: "d4717f23-7fe1-4c06-b411-0bb095e21361") : secret "cluster-monitoring-operator-tls" not found Jul 21 18:00:39.415724 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:39.415683 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q8fqf" event={"ID":"3257df56-1fd0-4f11-a9fa-b68d6daa1a1b","Type":"ContainerStarted","Data":"6c148384fd9edf55b9698d041f70704cb990a2b1418e7341423f9d609e4ca378"} Jul 21 18:00:39.443247 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:39.443042 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-q8fqf" podStartSLOduration=8.852553451 podStartE2EDuration="40.442887267s" totalImagesPullingTime="31.590333816s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:59:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:39.441170957 +0000 UTC m=+40.709482400" watchObservedRunningTime="2026-07-21 18:00:39.442887267 +0000 UTC m=+40.711198710" Jul 21 18:00:39.573947 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:39.573878 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:39.778499 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:39.778464 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:39.878501 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:39.878454 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:40.341272 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:40.341219 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:41.610947 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:41.610892 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:41.681521 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:41.681482 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:42.946513 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:42.946467 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:42.947031 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:42.946624 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:42.947031 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:42.946665 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle podName:043e03ca-7086-49e3-96b3-b824757c652a nodeName:}" failed. No retries permitted until 2026-07-21 18:00:50.946646005 +0000 UTC m=+52.214957424 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle") pod "router-default-799cfd6768-8vvdp" (UID: "043e03ca-7086-49e3-96b3-b824757c652a") : configmap references non-existent config key: service-ca.crt Jul 21 18:00:42.947031 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:42.946691 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:42.947031 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:42.946784 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 18:00:42.947031 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:42.946806 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6774fd47f-flh6l: secret "image-registry-tls" not found Jul 21 18:00:42.947031 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:42.946831 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 18:00:42.947031 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:42.946862 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls podName:8586960d-dd9a-45be-89e0-db7ab33d749c nodeName:}" failed. No retries permitted until 2026-07-21 18:00:50.94684432 +0000 UTC m=+52.215155739 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls") pod "image-registry-6774fd47f-flh6l" (UID: "8586960d-dd9a-45be-89e0-db7ab33d749c") : secret "image-registry-tls" not found Jul 21 18:00:42.947031 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:42.946881 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs podName:043e03ca-7086-49e3-96b3-b824757c652a nodeName:}" failed. No retries permitted until 2026-07-21 18:00:50.946872426 +0000 UTC m=+52.215183847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs") pod "router-default-799cfd6768-8vvdp" (UID: "043e03ca-7086-49e3-96b3-b824757c652a") : secret "router-metrics-certs-default" not found Jul 21 18:00:43.048133 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:43.048096 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:43.048327 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:43.048235 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gxlqf\" (UID: \"3761b47f-2110-4863-9db7-ffb3a826def2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" Jul 21 18:00:43.048327 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:43.048247 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 18:00:43.048327 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:43.048271 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:00:43.048327 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:43.048319 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls podName:befe4828-08d1-4685-84ef-766f46b64135 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:51.048298774 +0000 UTC m=+52.316610195 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls") pod "dns-default-krmfj" (UID: "befe4828-08d1-4685-84ef-766f46b64135") : secret "dns-default-metrics-tls" not found Jul 21 18:00:43.048556 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:43.048385 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 18:00:43.048556 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:43.048389 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 18:00:43.048556 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:43.048437 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls podName:d4717f23-7fe1-4c06-b411-0bb095e21361 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:51.048421502 +0000 UTC m=+52.316732940 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tvbh7" (UID: "d4717f23-7fe1-4c06-b411-0bb095e21361") : secret "cluster-monitoring-operator-tls" not found Jul 21 18:00:43.048556 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:43.048455 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls podName:3761b47f-2110-4863-9db7-ffb3a826def2 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:51.048445577 +0000 UTC m=+52.316757001 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gxlqf" (UID: "3761b47f-2110-4863-9db7-ffb3a826def2") : secret "samples-operator-tls" not found Jul 21 18:00:43.048556 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:43.048501 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-zvk8d\" (UID: \"8d00aa82-0b34-4f8b-a7a1-3c4e3458833f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" Jul 21 18:00:43.048806 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:43.048566 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 18:00:43.048806 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:43.048619 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert podName:8d00aa82-0b34-4f8b-a7a1-3c4e3458833f nodeName:}" failed. No retries permitted until 2026-07-21 18:00:51.048606065 +0000 UTC m=+52.316917503 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-zvk8d" (UID: "8d00aa82-0b34-4f8b-a7a1-3c4e3458833f") : secret "networking-console-plugin-cert" not found Jul 21 18:00:45.437179 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.437068 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cb5pk" event={"ID":"e0d15d58-ba71-482a-9cac-76f87f853a56","Type":"ContainerStarted","Data":"206b558cf03aaf773b88a607c4e2cea599fa1edec941a3616ed7d520c3293da8"} Jul 21 18:00:45.438820 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.438785 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" event={"ID":"cf634bfc-df89-4e85-874a-33bd28e7eb68","Type":"ContainerStarted","Data":"d0527652945e077ffcbb7a0c9a0835aa03e584484b3d2099cea4cdc8a4dcb6c9"} Jul 21 18:00:45.442078 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.442053 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/0.log" Jul 21 18:00:45.442223 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.442100 2568 generic.go:356] "Generic (PLEG): container finished" podID="7bc042d5-1297-4ac5-8237-76c5e2ab7ddd" containerID="2c4696d78384f2239cce694cb6e73dc65470f68f6cb16a6a9595dd1722370d6e" exitCode=255 Jul 21 18:00:45.442223 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.442180 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" event={"ID":"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd","Type":"ContainerDied","Data":"2c4696d78384f2239cce694cb6e73dc65470f68f6cb16a6a9595dd1722370d6e"} Jul 21 18:00:45.442837 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.442425 2568 scope.go:117] "RemoveContainer" containerID="2c4696d78384f2239cce694cb6e73dc65470f68f6cb16a6a9595dd1722370d6e" Jul 21 18:00:45.447106 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.446573 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" event={"ID":"46b0651b-2ded-4eee-b247-cf1404ba7ba3","Type":"ContainerStarted","Data":"4e397a4157fc19e7ba84f4c99e9ffc05fad4ac5c4884b207f05917a0d290bbff"} Jul 21 18:00:45.448680 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.448654 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ztxpr" event={"ID":"80cad807-9199-42c6-8b46-8d6887025525","Type":"ContainerStarted","Data":"b08192da38338f3f40b068cb976180c93d5a920c010899b6be4e23a2cdcbc1fb"} Jul 21 18:00:45.450786 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.450337 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vvvfz" event={"ID":"37231e4d-5ee2-44d9-9482-19501acca31d","Type":"ContainerStarted","Data":"b0ed4fb217174cd280453539c528dffb2c84b7c0b0c612b6f6af556d3e710c98"} Jul 21 18:00:45.452544 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.452136 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6jjkh" event={"ID":"d7074553-ba3e-43c7-8f4c-1037942bb3a9","Type":"ContainerStarted","Data":"9f56fe2db812a6189e8d9e1a2d41892e70af431795f23c8636c8d9064f8a7dcb"} Jul 21 18:00:45.461698 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.461493 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" podStartSLOduration=9.446539914 podStartE2EDuration="17.461470133s" totalImagesPullingTime="8.014930219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:45.459734451 +0000 UTC m=+46.728045897" watchObservedRunningTime="2026-07-21 18:00:45.461470133 +0000 UTC m=+46.729781556" Jul 21 18:00:45.481573 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.481291 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-ztxpr" podStartSLOduration=9.464560412 podStartE2EDuration="17.481270252s" totalImagesPullingTime="8.01670984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:45.479694374 +0000 UTC m=+46.748005829" watchObservedRunningTime="2026-07-21 18:00:45.481270252 +0000 UTC m=+46.749581696" Jul 21 18:00:45.499482 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.499416 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vvvfz" podStartSLOduration=2.477337006 podStartE2EDuration="10.49938224s" totalImagesPullingTime="8.022045234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:45.49867036 +0000 UTC m=+46.766981803" watchObservedRunningTime="2026-07-21 18:00:45.49938224 +0000 UTC m=+46.767693684" Jul 21 18:00:45.527777 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.527584 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" podStartSLOduration=8.513285214 podStartE2EDuration="16.527564002s" totalImagesPullingTime="8.014278788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:45.52567359 +0000 UTC m=+46.793985038" watchObservedRunningTime="2026-07-21 18:00:45.527564002 +0000 UTC m=+46.795875439" Jul 21 18:00:45.547729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.547524 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-6jjkh" podStartSLOduration=9.590615627 podStartE2EDuration="17.547503551s" totalImagesPullingTime="7.956887924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:45.546229162 +0000 UTC m=+46.814540606" watchObservedRunningTime="2026-07-21 18:00:45.547503551 +0000 UTC m=+46.815814994" Jul 21 18:00:45.553641 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.553607 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:45.553972 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.553930 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:45.672313 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.672282 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:45.736250 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:45.736185 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:46.461244 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:46.461165 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:00:46.462101 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:46.462082 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/0.log" Jul 21 18:00:46.462214 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:46.462120 2568 generic.go:356] "Generic (PLEG): container finished" podID="7bc042d5-1297-4ac5-8237-76c5e2ab7ddd" containerID="caf0ab7bfb389c4d901fac2a83d1f90b226ec6ee074ae7882666a535e72aee14" exitCode=255 Jul 21 18:00:46.462287 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:46.462232 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" event={"ID":"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd","Type":"ContainerDied","Data":"caf0ab7bfb389c4d901fac2a83d1f90b226ec6ee074ae7882666a535e72aee14"} Jul 21 18:00:46.462346 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:46.462302 2568 scope.go:117] "RemoveContainer" containerID="2c4696d78384f2239cce694cb6e73dc65470f68f6cb16a6a9595dd1722370d6e" Jul 21 18:00:46.462523 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:46.462502 2568 scope.go:117] "RemoveContainer" containerID="caf0ab7bfb389c4d901fac2a83d1f90b226ec6ee074ae7882666a535e72aee14" Jul 21 18:00:46.462720 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:46.462700 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-74hdc_openshift-console-operator(7bc042d5-1297-4ac5-8237-76c5e2ab7ddd)\"" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" podUID="7bc042d5-1297-4ac5-8237-76c5e2ab7ddd" Jul 21 18:00:46.503409 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:46.503354 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-cb5pk" podStartSLOduration=10.473137640000001 podStartE2EDuration="18.503332459s" totalImagesPullingTime="8.030194819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:46.501955813 +0000 UTC m=+47.770267261" watchObservedRunningTime="2026-07-21 18:00:46.503332459 +0000 UTC m=+47.771643901" Jul 21 18:00:46.986620 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:46.986578 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:47.125535 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.125475 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:47.438810 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.438772 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:47.444664 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.444631 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-swmjm"] Jul 21 18:00:47.468027 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.468003 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:00:47.468435 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.468339 2568 scope.go:117] "RemoveContainer" containerID="caf0ab7bfb389c4d901fac2a83d1f90b226ec6ee074ae7882666a535e72aee14" Jul 21 18:00:47.468508 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:47.468488 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-74hdc_openshift-console-operator(7bc042d5-1297-4ac5-8237-76c5e2ab7ddd)\"" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" podUID="7bc042d5-1297-4ac5-8237-76c5e2ab7ddd" Jul 21 18:00:47.485445 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.485410 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-swmjm"] Jul 21 18:00:47.485589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.485531 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-swmjm" Jul 21 18:00:47.488846 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.488816 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 21 18:00:47.489379 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.489356 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 21 18:00:47.489513 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.489383 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-q48wt\"" Jul 21 18:00:47.540195 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.540158 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:47.598790 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.598755 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bhwl\" (UniqueName: \"kubernetes.io/projected/79c07e15-05b5-4984-808b-f5ed1091b13f-kube-api-access-8bhwl\") pod \"migrator-7cc9c7ffd7-swmjm\" (UID: \"79c07e15-05b5-4984-808b-f5ed1091b13f\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-swmjm" Jul 21 18:00:47.690102 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.690014 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:47.700274 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.700231 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bhwl\" (UniqueName: \"kubernetes.io/projected/79c07e15-05b5-4984-808b-f5ed1091b13f-kube-api-access-8bhwl\") pod \"migrator-7cc9c7ffd7-swmjm\" (UID: \"79c07e15-05b5-4984-808b-f5ed1091b13f\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-swmjm" Jul 21 18:00:47.737896 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.737819 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bhwl\" (UniqueName: \"kubernetes.io/projected/79c07e15-05b5-4984-808b-f5ed1091b13f-kube-api-access-8bhwl\") pod \"migrator-7cc9c7ffd7-swmjm\" (UID: \"79c07e15-05b5-4984-808b-f5ed1091b13f\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-swmjm" Jul 21 18:00:47.739887 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.739855 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7gszs"] Jul 21 18:00:47.770363 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.770331 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7gszs"] Jul 21 18:00:47.770503 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.770430 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:47.773803 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.773778 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 21 18:00:47.774009 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.773993 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gzbvh\"" Jul 21 18:00:47.774623 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.774611 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 21 18:00:47.802861 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.802816 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:47.807713 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.807684 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-swmjm" Jul 21 18:00:47.882267 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.882230 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:47.902252 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.902208 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:47.902431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.902285 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/afe50d97-edb0-4de3-bf15-e189d083fb5d-crio-socket\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:47.902431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.902357 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/afe50d97-edb0-4de3-bf15-e189d083fb5d-data-volume\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:47.902527 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.902454 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:47.902527 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.902507 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzfpq\" (UniqueName: \"kubernetes.io/projected/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-api-access-bzfpq\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:47.962241 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:47.962219 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-swmjm"] Jul 21 18:00:47.964639 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:00:47.964603 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c07e15_05b5_4984_808b_f5ed1091b13f.slice/crio-4f5f190f20c4c990cf1691df0046f878e4eb6c24c950d3551a5b7d2ff58fe52a WatchSource:0}: Error finding container 4f5f190f20c4c990cf1691df0046f878e4eb6c24c950d3551a5b7d2ff58fe52a: Status 404 returned error can't find the container with id 4f5f190f20c4c990cf1691df0046f878e4eb6c24c950d3551a5b7d2ff58fe52a Jul 21 18:00:48.003850 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:48.003808 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/afe50d97-edb0-4de3-bf15-e189d083fb5d-crio-socket\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:48.004068 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:48.003873 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/afe50d97-edb0-4de3-bf15-e189d083fb5d-data-volume\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:48.004068 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:48.003981 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:48.004068 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:48.004010 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzfpq\" (UniqueName: \"kubernetes.io/projected/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-api-access-bzfpq\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:48.004068 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:48.004016 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/afe50d97-edb0-4de3-bf15-e189d083fb5d-crio-socket\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:48.004245 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:48.004095 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 18:00:48.004245 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:48.004156 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:48.004245 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:48.004166 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm podName:afe50d97-edb0-4de3-bf15-e189d083fb5d nodeName:}" failed. No retries permitted until 2026-07-21 18:00:48.504146478 +0000 UTC m=+49.772457897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7gszs" (UID: "afe50d97-edb0-4de3-bf15-e189d083fb5d") : configmap "kube-rbac-proxy" not found Jul 21 18:00:48.004350 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:48.004283 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/afe50d97-edb0-4de3-bf15-e189d083fb5d-data-volume\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:48.004350 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:48.004290 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 18:00:48.004410 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:48.004375 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls podName:afe50d97-edb0-4de3-bf15-e189d083fb5d nodeName:}" failed. No retries permitted until 2026-07-21 18:00:48.504350961 +0000 UTC m=+49.772662395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7gszs" (UID: "afe50d97-edb0-4de3-bf15-e189d083fb5d") : secret "insights-runtime-extractor-tls" not found Jul 21 18:00:48.014261 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:48.014228 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzfpq\" (UniqueName: \"kubernetes.io/projected/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-api-access-bzfpq\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:48.472871 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:48.472833 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-swmjm" event={"ID":"79c07e15-05b5-4984-808b-f5ed1091b13f","Type":"ContainerStarted","Data":"4f5f190f20c4c990cf1691df0046f878e4eb6c24c950d3551a5b7d2ff58fe52a"} Jul 21 18:00:48.509486 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:48.509444 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:48.509663 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:48.509570 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:48.509663 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:48.509601 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 18:00:48.509663 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:48.509645 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 18:00:48.509784 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:48.509671 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls podName:afe50d97-edb0-4de3-bf15-e189d083fb5d nodeName:}" failed. No retries permitted until 2026-07-21 18:00:49.509655513 +0000 UTC m=+50.777966932 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7gszs" (UID: "afe50d97-edb0-4de3-bf15-e189d083fb5d") : secret "insights-runtime-extractor-tls" not found Jul 21 18:00:48.509784 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:48.509717 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm podName:afe50d97-edb0-4de3-bf15-e189d083fb5d nodeName:}" failed. No retries permitted until 2026-07-21 18:00:49.509702469 +0000 UTC m=+50.778013890 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7gszs" (UID: "afe50d97-edb0-4de3-bf15-e189d083fb5d") : configmap "kube-rbac-proxy" not found Jul 21 18:00:48.586511 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:48.586469 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:48.728729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:48.728625 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:49.096693 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:49.096660 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:49.184141 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:49.184108 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:49.308671 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:49.308631 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:49.519759 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:49.519672 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:49.520289 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:49.519820 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 18:00:49.520289 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:49.519856 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:49.520289 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:49.519898 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls podName:afe50d97-edb0-4de3-bf15-e189d083fb5d nodeName:}" failed. No retries permitted until 2026-07-21 18:00:51.519874622 +0000 UTC m=+52.788186075 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7gszs" (UID: "afe50d97-edb0-4de3-bf15-e189d083fb5d") : secret "insights-runtime-extractor-tls" not found Jul 21 18:00:49.520289 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:49.519925 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 18:00:49.520289 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:49.519997 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm podName:afe50d97-edb0-4de3-bf15-e189d083fb5d nodeName:}" failed. No retries permitted until 2026-07-21 18:00:51.519979078 +0000 UTC m=+52.788290522 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7gszs" (UID: "afe50d97-edb0-4de3-bf15-e189d083fb5d") : configmap "kube-rbac-proxy" not found Jul 21 18:00:49.539525 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:49.539492 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:50.308581 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:50.308554 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vlkxx_dc216396-5088-4fd3-9409-f525712b3b92/dns-node-resolver/0.log" Jul 21 18:00:50.439560 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:50.439472 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:50.481593 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:50.481559 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-swmjm" event={"ID":"79c07e15-05b5-4984-808b-f5ed1091b13f","Type":"ContainerStarted","Data":"37f2df4029c89f6d994436b651d06093a94e4fc6a74c30d76b6ec8c5b30d4ef9"} Jul 21 18:00:50.481765 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:50.481619 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-swmjm" event={"ID":"79c07e15-05b5-4984-808b-f5ed1091b13f","Type":"ContainerStarted","Data":"abfdf9c2edae3dac1d42a25c69d654d6435cad808d7a2b578656afcae77eaf7a"} Jul 21 18:00:50.501204 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:50.501151 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-swmjm" podStartSLOduration=1.292406263 podStartE2EDuration="3.501131753s" totalImagesPullingTime="2.20872549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:50.499381492 +0000 UTC m=+51.767692935" watchObservedRunningTime="2026-07-21 18:00:50.501131753 +0000 UTC m=+51.769443197" Jul 21 18:00:50.906651 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:50.906623 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-44h8t_163da31b-27c0-4e33-b898-4a2066ccea13/node-ca/0.log" Jul 21 18:00:51.033895 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:51.033854 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:00:51.034105 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:51.033906 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:51.034105 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:51.033996 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:00:51.034105 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.034040 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 18:00:51.034105 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.034062 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6774fd47f-flh6l: secret "image-registry-tls" not found Jul 21 18:00:51.034261 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.034046 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 18:00:51.034261 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.034118 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls podName:8586960d-dd9a-45be-89e0-db7ab33d749c nodeName:}" failed. No retries permitted until 2026-07-21 18:01:07.034101243 +0000 UTC m=+68.302412664 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls") pod "image-registry-6774fd47f-flh6l" (UID: "8586960d-dd9a-45be-89e0-db7ab33d749c") : secret "image-registry-tls" not found Jul 21 18:00:51.034261 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.034186 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle podName:043e03ca-7086-49e3-96b3-b824757c652a nodeName:}" failed. No retries permitted until 2026-07-21 18:01:07.034168721 +0000 UTC m=+68.302480141 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle") pod "router-default-799cfd6768-8vvdp" (UID: "043e03ca-7086-49e3-96b3-b824757c652a") : configmap references non-existent config key: service-ca.crt Jul 21 18:00:51.034261 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.034200 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs podName:043e03ca-7086-49e3-96b3-b824757c652a nodeName:}" failed. No retries permitted until 2026-07-21 18:01:07.03419281 +0000 UTC m=+68.302504230 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs") pod "router-default-799cfd6768-8vvdp" (UID: "043e03ca-7086-49e3-96b3-b824757c652a") : secret "router-metrics-certs-default" not found Jul 21 18:00:51.134832 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:51.134793 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gxlqf\" (UID: \"3761b47f-2110-4863-9db7-ffb3a826def2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" Jul 21 18:00:51.134832 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:51.134839 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:00:51.135080 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:51.134888 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-zvk8d\" (UID: \"8d00aa82-0b34-4f8b-a7a1-3c4e3458833f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" Jul 21 18:00:51.135080 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:51.134914 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:00:51.135080 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.134985 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 18:00:51.135080 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.135023 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 18:00:51.135080 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.135042 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 18:00:51.135080 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.135023 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 18:00:51.135080 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.135060 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls podName:3761b47f-2110-4863-9db7-ffb3a826def2 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:07.135038415 +0000 UTC m=+68.403349840 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-gxlqf" (UID: "3761b47f-2110-4863-9db7-ffb3a826def2") : secret "samples-operator-tls" not found Jul 21 18:00:51.135402 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.135089 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert podName:8d00aa82-0b34-4f8b-a7a1-3c4e3458833f nodeName:}" failed. No retries permitted until 2026-07-21 18:01:07.135078525 +0000 UTC m=+68.403389944 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-zvk8d" (UID: "8d00aa82-0b34-4f8b-a7a1-3c4e3458833f") : secret "networking-console-plugin-cert" not found Jul 21 18:00:51.135402 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.135104 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls podName:befe4828-08d1-4685-84ef-766f46b64135 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:07.135095267 +0000 UTC m=+68.403406687 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls") pod "dns-default-krmfj" (UID: "befe4828-08d1-4685-84ef-766f46b64135") : secret "dns-default-metrics-tls" not found Jul 21 18:00:51.135402 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.135122 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls podName:d4717f23-7fe1-4c06-b411-0bb095e21361 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:07.135116479 +0000 UTC m=+68.403427899 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tvbh7" (UID: "d4717f23-7fe1-4c06-b411-0bb095e21361") : secret "cluster-monitoring-operator-tls" not found Jul 21 18:00:51.538749 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:51.538708 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:51.538931 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.538880 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 18:00:51.539012 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.538967 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls podName:afe50d97-edb0-4de3-bf15-e189d083fb5d nodeName:}" failed. No retries permitted until 2026-07-21 18:00:55.538949945 +0000 UTC m=+56.807261379 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7gszs" (UID: "afe50d97-edb0-4de3-bf15-e189d083fb5d") : secret "insights-runtime-extractor-tls" not found Jul 21 18:00:51.539012 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:51.539004 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:51.539085 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.539061 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 18:00:51.539117 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:51.539093 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm podName:afe50d97-edb0-4de3-bf15-e189d083fb5d nodeName:}" failed. No retries permitted until 2026-07-21 18:00:55.539084532 +0000 UTC m=+56.807395955 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7gszs" (UID: "afe50d97-edb0-4de3-bf15-e189d083fb5d") : configmap "kube-rbac-proxy" not found Jul 21 18:00:51.905811 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:51.905778 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vvvfz_37231e4d-5ee2-44d9-9482-19501acca31d/serve-healthcheck-canary/0.log" Jul 21 18:00:52.110472 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:52.110437 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-swmjm_79c07e15-05b5-4984-808b-f5ed1091b13f/migrator/0.log" Jul 21 18:00:52.148950 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:52.148912 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:52.224607 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:52.224498 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:52.313752 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:52.313718 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-swmjm_79c07e15-05b5-4984-808b-f5ed1091b13f/graceful-termination/0.log" Jul 21 18:00:52.507885 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:52.507805 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-llddq_cf634bfc-df89-4e85-874a-33bd28e7eb68/kube-storage-version-migrator-operator/0.log" Jul 21 18:00:53.024720 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:53.024687 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:54.920476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:54.920437 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:55.025384 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:55.025350 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:55.122308 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:55.122270 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:55.453689 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:55.453649 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:55.553908 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:55.553861 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:55.554102 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:55.553927 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:00:55.554307 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:55.554294 2568 scope.go:117] "RemoveContainer" containerID="caf0ab7bfb389c4d901fac2a83d1f90b226ec6ee074ae7882666a535e72aee14" Jul 21 18:00:55.554471 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:55.554450 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-74hdc_openshift-console-operator(7bc042d5-1297-4ac5-8237-76c5e2ab7ddd)\"" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" podUID="7bc042d5-1297-4ac5-8237-76c5e2ab7ddd" Jul 21 18:00:55.578650 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:55.578617 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:55.578816 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:55.578719 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:55.578816 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:55.578791 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 18:00:55.578924 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:00:55.578873 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls podName:afe50d97-edb0-4de3-bf15-e189d083fb5d nodeName:}" failed. No retries permitted until 2026-07-21 18:01:03.578850126 +0000 UTC m=+64.847161566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7gszs" (UID: "afe50d97-edb0-4de3-bf15-e189d083fb5d") : secret "insights-runtime-extractor-tls" not found Jul 21 18:00:55.579296 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:55.579275 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afe50d97-edb0-4de3-bf15-e189d083fb5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:00:56.148242 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:56.148201 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:56.269745 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:56.269704 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:56.406269 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:56.406190 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:56.600151 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:56.600100 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:56.704463 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:56.704365 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:57.398690 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:57.398654 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:57.499632 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:57.499598 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:57.599413 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:57.599382 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:57.794716 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:57.794618 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:58.399682 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:58.399648 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:58.591108 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:58.591078 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:58.795087 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:58.795009 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:58.861702 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:58.861662 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:00:59.200951 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:00:59.200901 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:00.513495 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:00.513458 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:00.615098 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:00.615060 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:00.795075 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:00.794988 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:01.209839 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:01.209794 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:01.285423 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:01.285369 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:01.379921 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:01.379896 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7h7m4" Jul 21 18:01:01.593509 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:01.593472 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:01.995366 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:01.995285 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:02.397894 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:02.397863 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:02.804546 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:02.804463 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:03.654265 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:03.654232 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:01:03.656552 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:03.656529 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afe50d97-edb0-4de3-bf15-e189d083fb5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7gszs\" (UID: \"afe50d97-edb0-4de3-bf15-e189d083fb5d\") " pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:01:03.683200 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:03.683170 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gzbvh\"" Jul 21 18:01:03.691230 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:03.691208 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7gszs" Jul 21 18:01:03.821880 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:03.821855 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7gszs"] Jul 21 18:01:03.824427 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:01:03.824396 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafe50d97_edb0_4de3_bf15_e189d083fb5d.slice/crio-b306c52c7e0baf606f996f83606bebc0c3eb3f7a21536d1f14f5479b24728104 WatchSource:0}: Error finding container b306c52c7e0baf606f996f83606bebc0c3eb3f7a21536d1f14f5479b24728104: Status 404 returned error can't find the container with id b306c52c7e0baf606f996f83606bebc0c3eb3f7a21536d1f14f5479b24728104 Jul 21 18:01:04.186652 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:04.186622 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:04.285842 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:04.285815 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:04.525615 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:04.525535 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7gszs" event={"ID":"afe50d97-edb0-4de3-bf15-e189d083fb5d","Type":"ContainerStarted","Data":"b306c52c7e0baf606f996f83606bebc0c3eb3f7a21536d1f14f5479b24728104"} Jul 21 18:01:04.595156 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:04.595116 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:04.866832 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:04.866799 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs\") pod \"network-metrics-daemon-kr8ht\" (UID: \"89ffba65-3615-478c-856d-a75629de4606\") " pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:01:04.869475 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:04.869457 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 18:01:04.879704 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:04.879673 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ffba65-3615-478c-856d-a75629de4606-metrics-certs\") pod \"network-metrics-daemon-kr8ht\" (UID: \"89ffba65-3615-478c-856d-a75629de4606\") " pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:01:04.967893 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:04.967851 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:01:04.971507 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:04.971488 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 18:01:04.980823 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:04.980785 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f9411eb0-f8da-4954-8df7-8fa2df70b0f0-original-pull-secret\") pod \"global-pull-secret-syncer-hrk92\" (UID: \"f9411eb0-f8da-4954-8df7-8fa2df70b0f0\") " pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:01:05.032925 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.032895 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hrk92" Jul 21 18:01:05.042679 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.041608 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jg5zx\"" Jul 21 18:01:05.049671 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.049551 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kr8ht" Jul 21 18:01:05.069324 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.069289 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gqgn\" (UniqueName: \"kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn\") pod \"network-check-target-4582q\" (UID: \"6f10df5a-b0d9-49b7-b495-247456b8cdd8\") " pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:01:05.071921 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.071888 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gqgn\" (UniqueName: \"kubernetes.io/projected/6f10df5a-b0d9-49b7-b495-247456b8cdd8-kube-api-access-5gqgn\") pod \"network-check-target-4582q\" (UID: \"6f10df5a-b0d9-49b7-b495-247456b8cdd8\") " pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:01:05.175816 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.175789 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hrk92"] Jul 21 18:01:05.178324 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:01:05.178302 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9411eb0_f8da_4954_8df7_8fa2df70b0f0.slice/crio-d75313fa1cecd34a9906eabcd37978442e92849dbf83c59d8ddb0a6b4bcf6db3 WatchSource:0}: Error finding container d75313fa1cecd34a9906eabcd37978442e92849dbf83c59d8ddb0a6b4bcf6db3: Status 404 returned error can't find the container with id d75313fa1cecd34a9906eabcd37978442e92849dbf83c59d8ddb0a6b4bcf6db3 Jul 21 18:01:05.190902 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.190869 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kr8ht"] Jul 21 18:01:05.194559 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.194523 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:05.195058 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:01:05.195014 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ffba65_3615_478c_856d_a75629de4606.slice/crio-6ebcd984d42534b56ae1b4f3024eb66e6c9d73db12f9cbb16a61f154a1679d9a WatchSource:0}: Error finding container 6ebcd984d42534b56ae1b4f3024eb66e6c9d73db12f9cbb16a61f154a1679d9a: Status 404 returned error can't find the container with id 6ebcd984d42534b56ae1b4f3024eb66e6c9d73db12f9cbb16a61f154a1679d9a Jul 21 18:01:05.328341 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.328306 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vznt9\"" Jul 21 18:01:05.336618 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.336591 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:01:05.400505 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.400474 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:05.493800 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.493590 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4582q"] Jul 21 18:01:05.496664 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:01:05.496632 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f10df5a_b0d9_49b7_b495_247456b8cdd8.slice/crio-d8e8d8be8fa35c72d095ff6dcd16f058eb26f811ff34b69c1e223e0f4eca1235 WatchSource:0}: Error finding container d8e8d8be8fa35c72d095ff6dcd16f058eb26f811ff34b69c1e223e0f4eca1235: Status 404 returned error can't find the container with id d8e8d8be8fa35c72d095ff6dcd16f058eb26f811ff34b69c1e223e0f4eca1235 Jul 21 18:01:05.530106 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.530065 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kr8ht" event={"ID":"89ffba65-3615-478c-856d-a75629de4606","Type":"ContainerStarted","Data":"6ebcd984d42534b56ae1b4f3024eb66e6c9d73db12f9cbb16a61f154a1679d9a"} Jul 21 18:01:05.531395 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.531365 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hrk92" event={"ID":"f9411eb0-f8da-4954-8df7-8fa2df70b0f0","Type":"ContainerStarted","Data":"d75313fa1cecd34a9906eabcd37978442e92849dbf83c59d8ddb0a6b4bcf6db3"} Jul 21 18:01:05.532978 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.532953 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4582q" event={"ID":"6f10df5a-b0d9-49b7-b495-247456b8cdd8","Type":"ContainerStarted","Data":"d8e8d8be8fa35c72d095ff6dcd16f058eb26f811ff34b69c1e223e0f4eca1235"} Jul 21 18:01:05.591902 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:05.591863 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:06.213992 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:06.213656 2568 scope.go:117] "RemoveContainer" containerID="caf0ab7bfb389c4d901fac2a83d1f90b226ec6ee074ae7882666a535e72aee14" Jul 21 18:01:06.542253 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:06.541118 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4582q" event={"ID":"6f10df5a-b0d9-49b7-b495-247456b8cdd8","Type":"ContainerStarted","Data":"d5decaca50827f949bc56cafbe86e2e89e6bf2242cde030eb4429a6f68e1365c"} Jul 21 18:01:06.542253 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:06.542014 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:01:06.547784 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:06.547140 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:01:06.547784 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:06.547255 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" event={"ID":"7bc042d5-1297-4ac5-8237-76c5e2ab7ddd","Type":"ContainerStarted","Data":"4e3156677bdacada7c8e1da37092e3c4397117821fe88d8731ee89c777efbbdf"} Jul 21 18:01:06.547784 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:06.547561 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:01:06.550191 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:06.550116 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kr8ht" event={"ID":"89ffba65-3615-478c-856d-a75629de4606","Type":"ContainerStarted","Data":"058d912e74784e588875932cfb0962b3ca6885050319e6886daf9f49b20ab278"} Jul 21 18:01:06.561127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:06.560591 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4582q" podStartSLOduration=67.560569362 podStartE2EDuration="1m7.560569362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:59:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:06.559681597 +0000 UTC m=+67.827993041" watchObservedRunningTime="2026-07-21 18:01:06.560569362 +0000 UTC m=+67.828880806" Jul 21 18:01:06.582863 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:06.581721 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" podStartSLOduration=30.561341072 podStartE2EDuration="38.581698069s" totalImagesPullingTime="8.020356997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:06.581162586 +0000 UTC m=+67.849474027" watchObservedRunningTime="2026-07-21 18:01:06.581698069 +0000 UTC m=+67.850009512" Jul 21 18:01:07.091038 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.090761 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:01:07.091038 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.090907 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:01:07.091038 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.090960 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:01:07.091571 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.091518 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/043e03ca-7086-49e3-96b3-b824757c652a-service-ca-bundle\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:01:07.094456 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.094428 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/043e03ca-7086-49e3-96b3-b824757c652a-metrics-certs\") pod \"router-default-799cfd6768-8vvdp\" (UID: \"043e03ca-7086-49e3-96b3-b824757c652a\") " pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:01:07.094571 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.094428 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls\") pod \"image-registry-6774fd47f-flh6l\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:01:07.191809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.191767 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gxlqf\" (UID: \"3761b47f-2110-4863-9db7-ffb3a826def2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" Jul 21 18:01:07.192027 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.191824 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:01:07.192027 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.191871 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-zvk8d\" (UID: \"8d00aa82-0b34-4f8b-a7a1-3c4e3458833f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" Jul 21 18:01:07.192027 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.191908 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:01:07.195261 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.195201 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/befe4828-08d1-4685-84ef-766f46b64135-metrics-tls\") pod \"dns-default-krmfj\" (UID: \"befe4828-08d1-4685-84ef-766f46b64135\") " pod="openshift-dns/dns-default-krmfj" Jul 21 18:01:07.195863 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.195814 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3761b47f-2110-4863-9db7-ffb3a826def2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-gxlqf\" (UID: \"3761b47f-2110-4863-9db7-ffb3a826def2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" Jul 21 18:01:07.196322 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.196297 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d00aa82-0b34-4f8b-a7a1-3c4e3458833f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-zvk8d\" (UID: \"8d00aa82-0b34-4f8b-a7a1-3c4e3458833f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" Jul 21 18:01:07.197529 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.197507 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4717f23-7fe1-4c06-b411-0bb095e21361-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tvbh7\" (UID: \"d4717f23-7fe1-4c06-b411-0bb095e21361\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:01:07.254001 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.253967 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-74hdc" Jul 21 18:01:07.271574 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.271536 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7t5gz\"" Jul 21 18:01:07.277114 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.277082 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xvdbv\"" Jul 21 18:01:07.278633 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.278592 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:01:07.285428 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.285405 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:01:07.312583 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.312504 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-lq89g\"" Jul 21 18:01:07.320240 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.320204 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" Jul 21 18:01:07.331768 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.331704 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-d59xs\"" Jul 21 18:01:07.336994 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.336732 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" Jul 21 18:01:07.352930 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.352819 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pqxk6\"" Jul 21 18:01:07.359367 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.359112 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-krmfj" Jul 21 18:01:07.368877 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.368843 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-c98vg\"" Jul 21 18:01:07.373522 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.373486 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" Jul 21 18:01:07.503851 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.503817 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6774fd47f-flh6l"] Jul 21 18:01:07.524544 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.524513 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-799cfd6768-8vvdp"] Jul 21 18:01:07.538900 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.538763 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:07.557312 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.557266 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kr8ht" event={"ID":"89ffba65-3615-478c-856d-a75629de4606","Type":"ContainerStarted","Data":"cfedbda6df48e6404a2165f04d50f748fb13801737f7816f90c5992f20bcdeb1"} Jul 21 18:01:07.598022 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.597964 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kr8ht" podStartSLOduration=67.477192863 podStartE2EDuration="1m8.597919007s" totalImagesPullingTime="1.120726144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:07.597769871 +0000 UTC m=+68.866081315" watchObservedRunningTime="2026-07-21 18:01:07.597919007 +0000 UTC m=+68.866230452" Jul 21 18:01:07.655776 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.655691 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:07.947866 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:07.947757 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:08.128450 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:08.128415 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:09.146303 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:09.146256 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:09.219055 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:09.219015 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:09.351581 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:09.351535 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:09.550533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:09.550448 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:09.773516 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:09.773460 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:09.876986 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:09.876914 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:10.024107 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:10.024032 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:11.196696 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:01:11.196655 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod043e03ca_7086_49e3_96b3_b824757c652a.slice/crio-f165e9d96f5a0bd4305330e4eed852ecc88289dae926509fdef15347620bf4db WatchSource:0}: Error finding container f165e9d96f5a0bd4305330e4eed852ecc88289dae926509fdef15347620bf4db: Status 404 returned error can't find the container with id f165e9d96f5a0bd4305330e4eed852ecc88289dae926509fdef15347620bf4db Jul 21 18:01:11.324796 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:01:11.324754 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8586960d_dd9a_45be_89e0_db7ab33d749c.slice/crio-6c0d71dcbf22f1265554bc9508937f41894bafbc150bbabfc0dadbc785d2b852 WatchSource:0}: Error finding container 6c0d71dcbf22f1265554bc9508937f41894bafbc150bbabfc0dadbc785d2b852: Status 404 returned error can't find the container with id 6c0d71dcbf22f1265554bc9508937f41894bafbc150bbabfc0dadbc785d2b852 Jul 21 18:01:11.466375 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:11.466296 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf"] Jul 21 18:01:11.574633 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:11.574593 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6774fd47f-flh6l" event={"ID":"8586960d-dd9a-45be-89e0-db7ab33d749c","Type":"ContainerStarted","Data":"6c0d71dcbf22f1265554bc9508937f41894bafbc150bbabfc0dadbc785d2b852"} Jul 21 18:01:11.576046 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:11.576012 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-799cfd6768-8vvdp" event={"ID":"043e03ca-7086-49e3-96b3-b824757c652a","Type":"ContainerStarted","Data":"f165e9d96f5a0bd4305330e4eed852ecc88289dae926509fdef15347620bf4db"} Jul 21 18:01:11.624405 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:11.624365 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:11.729687 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:11.729591 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:12.389599 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.389565 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:12.455319 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.455067 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-xmhx6"] Jul 21 18:01:12.482776 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.481962 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-xmhx6"] Jul 21 18:01:12.482776 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.482100 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-xmhx6" Jul 21 18:01:12.485727 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.485110 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-fmzlz\"" Jul 21 18:01:12.485727 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.485363 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 21 18:01:12.489628 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.489305 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 21 18:01:12.506158 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.506064 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:12.535828 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.535183 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lmw8\" (UniqueName: \"kubernetes.io/projected/d23f3bbd-bf10-4a67-853d-b8d2f6a875ae-kube-api-access-5lmw8\") pod \"downloads-5bc67dcd46-xmhx6\" (UID: \"d23f3bbd-bf10-4a67-853d-b8d2f6a875ae\") " pod="openshift-console/downloads-5bc67dcd46-xmhx6" Jul 21 18:01:12.585738 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.585639 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" event={"ID":"3761b47f-2110-4863-9db7-ffb3a826def2","Type":"ContainerStarted","Data":"6eab3cd9375bee207e056865c5a0dff0dda34e46166880d1f20feace91e7189a"} Jul 21 18:01:12.594178 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.593611 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6774fd47f-flh6l" event={"ID":"8586960d-dd9a-45be-89e0-db7ab33d749c","Type":"ContainerStarted","Data":"f8b76fe184462e859c7b1a5a40a879d812d2ccaaff69eca02b995506540d107a"} Jul 21 18:01:12.594178 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.593753 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:01:12.597465 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.596330 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-799cfd6768-8vvdp" event={"ID":"043e03ca-7086-49e3-96b3-b824757c652a","Type":"ContainerStarted","Data":"9bf5239e3c99e3be0aa9e8761d409c6c1cc33ae4146cdba68163d1bf4773b421"} Jul 21 18:01:12.631455 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.631420 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:12.636772 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.636731 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lmw8\" (UniqueName: \"kubernetes.io/projected/d23f3bbd-bf10-4a67-853d-b8d2f6a875ae-kube-api-access-5lmw8\") pod \"downloads-5bc67dcd46-xmhx6\" (UID: \"d23f3bbd-bf10-4a67-853d-b8d2f6a875ae\") " pod="openshift-console/downloads-5bc67dcd46-xmhx6" Jul 21 18:01:12.651968 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.651902 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6774fd47f-flh6l" podStartSLOduration=49.6518817 podStartE2EDuration="49.6518817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:12.6236365 +0000 UTC m=+73.891947945" watchObservedRunningTime="2026-07-21 18:01:12.6518817 +0000 UTC m=+73.920193142" Jul 21 18:01:12.652123 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.652102 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7"] Jul 21 18:01:12.660408 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.660373 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lmw8\" (UniqueName: \"kubernetes.io/projected/d23f3bbd-bf10-4a67-853d-b8d2f6a875ae-kube-api-access-5lmw8\") pod \"downloads-5bc67dcd46-xmhx6\" (UID: \"d23f3bbd-bf10-4a67-853d-b8d2f6a875ae\") " pod="openshift-console/downloads-5bc67dcd46-xmhx6" Jul 21 18:01:12.671576 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.671525 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-799cfd6768-8vvdp" podStartSLOduration=44.671504624 podStartE2EDuration="44.671504624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:12.671429183 +0000 UTC m=+73.939740648" watchObservedRunningTime="2026-07-21 18:01:12.671504624 +0000 UTC m=+73.939816066" Jul 21 18:01:12.688283 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.688075 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d"] Jul 21 18:01:12.689406 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.689384 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-krmfj"] Jul 21 18:01:12.804334 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.804302 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-xmhx6" Jul 21 18:01:12.843100 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.843060 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:12.945110 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.945019 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:12.990062 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:12.990031 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-xmhx6"] Jul 21 18:01:12.993210 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:01:12.993180 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23f3bbd_bf10_4a67_853d_b8d2f6a875ae.slice/crio-84fa148d1a5a9130b45f30d577cd827ff1c8e95cc50acd2c5a43761382030e22 WatchSource:0}: Error finding container 84fa148d1a5a9130b45f30d577cd827ff1c8e95cc50acd2c5a43761382030e22: Status 404 returned error can't find the container with id 84fa148d1a5a9130b45f30d577cd827ff1c8e95cc50acd2c5a43761382030e22 Jul 21 18:01:13.286592 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:13.286515 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:01:13.289303 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:13.289275 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:01:13.604513 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:13.604432 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" event={"ID":"d4717f23-7fe1-4c06-b411-0bb095e21361","Type":"ContainerStarted","Data":"f9568dd5b77fd4eb96a30a48d4a1bfc84e6ffb1c634af0f9c046a0f34610b8e9"} Jul 21 18:01:13.606499 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:13.606462 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hrk92" event={"ID":"f9411eb0-f8da-4954-8df7-8fa2df70b0f0","Type":"ContainerStarted","Data":"e9531773d776c5c4f1ebedf63c1d0b90450317fedfac9ad7a843bd3314a2e460"} Jul 21 18:01:13.608668 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:13.608595 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-krmfj" event={"ID":"befe4828-08d1-4685-84ef-766f46b64135","Type":"ContainerStarted","Data":"78c1403a94bb3a6c4b40a08b0c6d4a9d699e974e69798006350a6a439e2b76cf"} Jul 21 18:01:13.610759 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:13.610726 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7gszs" event={"ID":"afe50d97-edb0-4de3-bf15-e189d083fb5d","Type":"ContainerStarted","Data":"09332fbd66ab726cafd35405340ceadd941a544356a0d72182826921096af3bc"} Jul 21 18:01:13.613473 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:13.613445 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" event={"ID":"8d00aa82-0b34-4f8b-a7a1-3c4e3458833f","Type":"ContainerStarted","Data":"ef967c6bc707be54d1b3e0dfdb7221fad7a5991cf418a894c6111eb5b5435666"} Jul 21 18:01:13.615105 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:13.615076 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-xmhx6" event={"ID":"d23f3bbd-bf10-4a67-853d-b8d2f6a875ae","Type":"ContainerStarted","Data":"84fa148d1a5a9130b45f30d577cd827ff1c8e95cc50acd2c5a43761382030e22"} Jul 21 18:01:13.615522 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:13.615474 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:01:13.617045 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:13.617011 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-799cfd6768-8vvdp" Jul 21 18:01:13.629836 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:13.629774 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hrk92" podStartSLOduration=67.38373193 podStartE2EDuration="1m14.629754791s" totalImagesPullingTime="7.246022861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:13.628906803 +0000 UTC m=+74.897218264" watchObservedRunningTime="2026-07-21 18:01:13.629754791 +0000 UTC m=+74.898066211" Jul 21 18:01:14.415827 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:14.415780 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:14.518419 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:14.518327 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:17.783420 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.783377 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85bb7857d-2t948"] Jul 21 18:01:17.787021 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.786995 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.789439 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.789407 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 21 18:01:17.790294 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.790239 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hntdr\"" Jul 21 18:01:17.790294 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.790290 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 21 18:01:17.790486 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.790341 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 21 18:01:17.790486 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.790378 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 21 18:01:17.790486 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.790298 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 21 18:01:17.799059 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.799027 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85bb7857d-2t948"] Jul 21 18:01:17.880042 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.880002 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8eb60fee-4c59-4366-b90f-038b7708ed40-console-oauth-config\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.880220 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.880063 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-service-ca\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.880220 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.880115 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8eb60fee-4c59-4366-b90f-038b7708ed40-console-serving-cert\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.880220 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.880194 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6w2s\" (UniqueName: \"kubernetes.io/projected/8eb60fee-4c59-4366-b90f-038b7708ed40-kube-api-access-p6w2s\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.880364 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.880233 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-oauth-serving-cert\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.880364 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.880283 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-console-config\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.981321 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.981285 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6w2s\" (UniqueName: \"kubernetes.io/projected/8eb60fee-4c59-4366-b90f-038b7708ed40-kube-api-access-p6w2s\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.981496 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.981336 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-oauth-serving-cert\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.981622 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.981599 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-console-config\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.981725 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.981664 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8eb60fee-4c59-4366-b90f-038b7708ed40-console-oauth-config\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.981725 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.981704 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-service-ca\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.981838 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.981767 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8eb60fee-4c59-4366-b90f-038b7708ed40-console-serving-cert\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.982164 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.982109 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-oauth-serving-cert\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.982343 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.982320 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-console-config\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.982611 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.982592 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-service-ca\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.984388 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.984362 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8eb60fee-4c59-4366-b90f-038b7708ed40-console-oauth-config\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:17.984728 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:17.984704 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8eb60fee-4c59-4366-b90f-038b7708ed40-console-serving-cert\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:18.000900 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:18.000864 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6w2s\" (UniqueName: \"kubernetes.io/projected/8eb60fee-4c59-4366-b90f-038b7708ed40-kube-api-access-p6w2s\") pod \"console-85bb7857d-2t948\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:18.100184 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:18.100054 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:18.611817 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:18.611775 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:18.712707 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:18.712664 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:19.428856 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.428745 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85bb7857d-2t948"] Jul 21 18:01:19.643730 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.643647 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" event={"ID":"d4717f23-7fe1-4c06-b411-0bb095e21361","Type":"ContainerStarted","Data":"2c43475f911f3edd2c91400d52e6963495634242ef49f29a72ba87b6d765f0e5"} Jul 21 18:01:19.646530 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.646442 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-krmfj" event={"ID":"befe4828-08d1-4685-84ef-766f46b64135","Type":"ContainerStarted","Data":"54a1a32689516805d308106d2a6212d166f30c01e457f65b6afdbd8720f8e41c"} Jul 21 18:01:19.646530 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.646515 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-krmfj" Jul 21 18:01:19.646530 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.646540 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-krmfj" event={"ID":"befe4828-08d1-4685-84ef-766f46b64135","Type":"ContainerStarted","Data":"6a10a5b303164cebcff33a81c2fdd641953776243df64564ee33c872b4968f3f"} Jul 21 18:01:19.651020 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.650976 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7gszs" event={"ID":"afe50d97-edb0-4de3-bf15-e189d083fb5d","Type":"ContainerStarted","Data":"ab4c01f0214a7b189f09956fbe678acbcd274979fcfdfda7f28fc42aa4e6c041"} Jul 21 18:01:19.652987 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.652909 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85bb7857d-2t948" event={"ID":"8eb60fee-4c59-4366-b90f-038b7708ed40","Type":"ContainerStarted","Data":"2414b8741658e24781b787a33545671cabef31836d7a7e7bddea8ac38f1a587e"} Jul 21 18:01:19.655089 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.655060 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" event={"ID":"3761b47f-2110-4863-9db7-ffb3a826def2","Type":"ContainerStarted","Data":"1920f432e82e1abcdd04c6aee7e932aa448a1b57a739cfe36464ff63a7a69093"} Jul 21 18:01:19.655221 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.655113 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" event={"ID":"3761b47f-2110-4863-9db7-ffb3a826def2","Type":"ContainerStarted","Data":"acc78701ef27a65531e872d7223ac1f5d548ebb0793a37895ff8172b36128fde"} Jul 21 18:01:19.656807 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.656771 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" event={"ID":"8d00aa82-0b34-4f8b-a7a1-3c4e3458833f","Type":"ContainerStarted","Data":"10a4da678671ebef789c12352b4450be61845562e9f6bcd6695a849821600200"} Jul 21 18:01:19.677295 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.677176 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tvbh7" podStartSLOduration=44.174864741 podStartE2EDuration="50.677153831s" totalImagesPullingTime="6.50228909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:19.676650106 +0000 UTC m=+80.944961549" watchObservedRunningTime="2026-07-21 18:01:19.677153831 +0000 UTC m=+80.945465274" Jul 21 18:01:19.703683 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.703623 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-gxlqf" podStartSLOduration=44.670599059 podStartE2EDuration="50.703603575s" totalImagesPullingTime="6.033004516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:19.700828297 +0000 UTC m=+80.969139742" watchObservedRunningTime="2026-07-21 18:01:19.703603575 +0000 UTC m=+80.971915017" Jul 21 18:01:19.704400 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.704338 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:19.724927 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.724316 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-krmfj" podStartSLOduration=38.232615902 podStartE2EDuration="44.724295605s" totalImagesPullingTime="6.491679703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:19.723800103 +0000 UTC m=+80.992111548" watchObservedRunningTime="2026-07-21 18:01:19.724295605 +0000 UTC m=+80.992607048" Jul 21 18:01:19.745097 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:19.745024 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-zvk8d" podStartSLOduration=44.308592375 podStartE2EDuration="50.744999827s" totalImagesPullingTime="6.436407452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:19.743899503 +0000 UTC m=+81.012210950" watchObservedRunningTime="2026-07-21 18:01:19.744999827 +0000 UTC m=+81.013311272" Jul 21 18:01:20.765109 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:20.765009 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:20.838250 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:20.838120 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:21.664051 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:21.663548 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:21.748783 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:21.748734 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:24.614033 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:24.613980 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:24.710323 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:24.710218 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:26.619354 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:26.619308 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:26.766205 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:26.766166 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:26.880924 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:26.880837 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:26.978954 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:26.978900 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:27.588816 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:27.588779 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:27.691877 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:27.691829 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:27.926639 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:27.926590 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:28.032537 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:28.032490 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:28.109482 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:28.109444 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:29.519262 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:29.519205 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:29.599777 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:29.599696 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:29.664588 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:29.664555 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-krmfj" Jul 21 18:01:30.846426 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:30.846344 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85bb7857d-2t948"] Jul 21 18:01:31.768049 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:31.768017 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:33.111021 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.110992 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:33.203911 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.203879 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:33.326617 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.326571 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:33.399477 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.399445 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cg5ps"] Jul 21 18:01:33.416330 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.416278 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.419193 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.419169 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 21 18:01:33.419322 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.419171 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q5ldh\"" Jul 21 18:01:33.419322 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.419171 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 21 18:01:33.420226 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.420208 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 21 18:01:33.420318 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.420210 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 21 18:01:33.428556 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.428534 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:33.522992 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.522890 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f91d7dcc-7895-4302-998c-c26e296b5c85-sys\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.522992 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.522968 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-textfile\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.523190 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.522996 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f91d7dcc-7895-4302-998c-c26e296b5c85-metrics-client-ca\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.523190 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.523038 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-wtmp\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.523190 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.523065 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.523190 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.523104 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkxxd\" (UniqueName: \"kubernetes.io/projected/f91d7dcc-7895-4302-998c-c26e296b5c85-kube-api-access-dkxxd\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.523190 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.523128 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-tls\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.523402 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.523196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-accelerators-collector-config\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.523402 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.523252 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f91d7dcc-7895-4302-998c-c26e296b5c85-root\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.584773 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.584734 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:33.620532 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.620498 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:01:33.624300 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.624267 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f91d7dcc-7895-4302-998c-c26e296b5c85-root\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.624456 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.624322 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f91d7dcc-7895-4302-998c-c26e296b5c85-sys\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.624456 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.624399 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f91d7dcc-7895-4302-998c-c26e296b5c85-sys\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.624456 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.624401 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f91d7dcc-7895-4302-998c-c26e296b5c85-root\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.624456 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.624433 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-textfile\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.624658 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.624459 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f91d7dcc-7895-4302-998c-c26e296b5c85-metrics-client-ca\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.624658 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.624491 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-wtmp\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.624658 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.624524 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.624658 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.624563 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkxxd\" (UniqueName: \"kubernetes.io/projected/f91d7dcc-7895-4302-998c-c26e296b5c85-kube-api-access-dkxxd\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.624658 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.624587 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-tls\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.624658 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.624613 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-accelerators-collector-config\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.625076 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.624699 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-wtmp\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.625076 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.624776 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-textfile\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.625076 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:01:33.624897 2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 21 18:01:33.625076 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:01:33.624999 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-tls podName:f91d7dcc-7895-4302-998c-c26e296b5c85 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:34.124976911 +0000 UTC m=+95.393288340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-tls") pod "node-exporter-cg5ps" (UID: "f91d7dcc-7895-4302-998c-c26e296b5c85") : secret "node-exporter-tls" not found Jul 21 18:01:33.625301 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.625123 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f91d7dcc-7895-4302-998c-c26e296b5c85-metrics-client-ca\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.625301 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.625165 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-accelerators-collector-config\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.627388 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.627344 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.635776 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.635749 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkxxd\" (UniqueName: \"kubernetes.io/projected/f91d7dcc-7895-4302-998c-c26e296b5c85-kube-api-access-dkxxd\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:33.687392 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.687299 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:33.723742 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.723680 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7gszs" event={"ID":"afe50d97-edb0-4de3-bf15-e189d083fb5d","Type":"ContainerStarted","Data":"e0d1b01bc2fa848407b5f1958d4623d7e34bcc0632a46b4a4f9cf7feddc63aa7"} Jul 21 18:01:33.726357 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.726325 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85bb7857d-2t948" event={"ID":"8eb60fee-4c59-4366-b90f-038b7708ed40","Type":"ContainerStarted","Data":"525bff3c0984153ee9fb687f662632b864a7b0d5e092922352fa49a682d1f223"} Jul 21 18:01:33.728645 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.728617 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-xmhx6" event={"ID":"d23f3bbd-bf10-4a67-853d-b8d2f6a875ae","Type":"ContainerStarted","Data":"b32caab0566af34208238bcb26e430a9448c20242be2a8d4f8d1d2f55606967d"} Jul 21 18:01:33.728754 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.728671 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-xmhx6" Jul 21 18:01:33.739582 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.739558 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-xmhx6" Jul 21 18:01:33.774241 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.774148 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7gszs" podStartSLOduration=17.738487191 podStartE2EDuration="46.774131118s" totalImagesPullingTime="29.035643927s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:33.747247031 +0000 UTC m=+95.015558506" watchObservedRunningTime="2026-07-21 18:01:33.774131118 +0000 UTC m=+95.042442559" Jul 21 18:01:33.774413 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.774267 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-xmhx6" podStartSLOduration=1.615318029 podStartE2EDuration="21.774261606s" totalImagesPullingTime="20.158943577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:33.77277876 +0000 UTC m=+95.041090206" watchObservedRunningTime="2026-07-21 18:01:33.774261606 +0000 UTC m=+95.042573047" Jul 21 18:01:33.781787 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:33.781757 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:34.128555 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:34.128513 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-tls\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:34.131444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:34.131407 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f91d7dcc-7895-4302-998c-c26e296b5c85-node-exporter-tls\") pod \"node-exporter-cg5ps\" (UID: \"f91d7dcc-7895-4302-998c-c26e296b5c85\") " pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:34.325832 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:34.325781 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cg5ps" Jul 21 18:01:34.391384 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:01:34.391353 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf91d7dcc_7895_4302_998c_c26e296b5c85.slice/crio-b7a1ffd56c99ae8c8c9d670564dfddc954878403361c772af1bf2d3b1a164e66 WatchSource:0}: Error finding container b7a1ffd56c99ae8c8c9d670564dfddc954878403361c772af1bf2d3b1a164e66: Status 404 returned error can't find the container with id b7a1ffd56c99ae8c8c9d670564dfddc954878403361c772af1bf2d3b1a164e66 Jul 21 18:01:34.468711 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:34.468647 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85bb7857d-2t948" podStartSLOduration=3.798331314 podStartE2EDuration="17.468623225s" totalImagesPullingTime="13.670291911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:33.796793223 +0000 UTC m=+95.065104681" watchObservedRunningTime="2026-07-21 18:01:34.468623225 +0000 UTC m=+95.736934669" Jul 21 18:01:34.470528 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:34.470497 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6774fd47f-flh6l"] Jul 21 18:01:34.507479 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:34.507435 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:34.606290 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:34.606245 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:34.714474 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:34.714384 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:34.735052 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:34.735010 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cg5ps" event={"ID":"f91d7dcc-7895-4302-998c-c26e296b5c85","Type":"ContainerStarted","Data":"b7a1ffd56c99ae8c8c9d670564dfddc954878403361c772af1bf2d3b1a164e66"} Jul 21 18:01:34.978548 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:34.978458 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:35.086025 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:35.085950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:35.185833 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:35.185702 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:35.414189 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:35.414089 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:35.509663 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:35.509623 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:36.407045 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:36.406994 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:36.485451 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:36.485416 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:36.682408 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:36.682367 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:36.746022 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:36.745985 2568 generic.go:356] "Generic (PLEG): container finished" podID="f91d7dcc-7895-4302-998c-c26e296b5c85" containerID="040bf9f5abf5bd35e9637e9789af7d1489fc2ad844f20c864e1d30c6992ead98" exitCode=0 Jul 21 18:01:36.746206 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:36.746083 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cg5ps" event={"ID":"f91d7dcc-7895-4302-998c-c26e296b5c85","Type":"ContainerDied","Data":"040bf9f5abf5bd35e9637e9789af7d1489fc2ad844f20c864e1d30c6992ead98"} Jul 21 18:01:36.897959 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:36.897809 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:36.998264 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:36.998228 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:37.113776 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:37.113734 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:37.753105 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:37.753066 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cg5ps" event={"ID":"f91d7dcc-7895-4302-998c-c26e296b5c85","Type":"ContainerStarted","Data":"ef8f75ae95578bd543165d3985de61283e9d2493b250b3f839c7c8eaf20ac793"} Jul 21 18:01:37.753568 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:37.753129 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cg5ps" event={"ID":"f91d7dcc-7895-4302-998c-c26e296b5c85","Type":"ContainerStarted","Data":"76075e291cef0e665d036bf45198e224147cd4164aab6166f648c6c90873534d"} Jul 21 18:01:38.100841 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:38.100804 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:38.180850 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:38.180810 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:38.319118 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:38.319083 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:38.423411 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:38.423312 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:38.565044 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:38.565009 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4582q" Jul 21 18:01:38.589862 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:38.589805 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cg5ps" podStartSLOduration=3.801364333 podStartE2EDuration="5.589787975s" totalImagesPullingTime="1.674254518s" totalInitContainerRuntime="114.169124ms" isStatefulPod=true podCreationTimestamp="2026-07-21 18:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:37.78351032 +0000 UTC m=+99.051821762" watchObservedRunningTime="2026-07-21 18:01:38.589787975 +0000 UTC m=+99.858099535" Jul 21 18:01:38.784226 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:38.784123 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:38.878390 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:38.878354 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:40.625806 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:40.625761 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:40.725970 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:40.725915 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:42.276745 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:42.276693 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:42.349443 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:42.349400 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:42.723691 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:42.723655 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:42.787320 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:42.787287 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:43.156681 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:43.156644 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:43.226371 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:43.226336 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:47.940994 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:47.940958 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:48.052643 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:48.052611 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:49.181822 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:49.181790 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:49.275701 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:49.275666 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:49.390903 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:49.390873 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:51.807912 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:51.807872 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:51.905346 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:51.905309 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:56.623450 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:56.623403 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:57.264017 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:57.263981 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:01:58.753495 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:58.753453 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85bb7857d-2t948" podUID="8eb60fee-4c59-4366-b90f-038b7708ed40" containerName="console" containerID="cri-o://525bff3c0984153ee9fb687f662632b864a7b0d5e092922352fa49a682d1f223" gracePeriod=15 Jul 21 18:01:59.039826 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.039794 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:59.153548 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.153512 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6w2s\" (UniqueName: \"kubernetes.io/projected/8eb60fee-4c59-4366-b90f-038b7708ed40-kube-api-access-p6w2s\") pod \"8eb60fee-4c59-4366-b90f-038b7708ed40\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " Jul 21 18:01:59.153548 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.153548 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-oauth-serving-cert\") pod \"8eb60fee-4c59-4366-b90f-038b7708ed40\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " Jul 21 18:01:59.153767 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.153578 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-console-config\") pod \"8eb60fee-4c59-4366-b90f-038b7708ed40\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " Jul 21 18:01:59.153767 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.153629 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8eb60fee-4c59-4366-b90f-038b7708ed40-console-oauth-config\") pod \"8eb60fee-4c59-4366-b90f-038b7708ed40\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " Jul 21 18:01:59.153767 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.153692 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-service-ca\") pod \"8eb60fee-4c59-4366-b90f-038b7708ed40\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " Jul 21 18:01:59.153767 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.153734 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8eb60fee-4c59-4366-b90f-038b7708ed40-console-serving-cert\") pod \"8eb60fee-4c59-4366-b90f-038b7708ed40\" (UID: \"8eb60fee-4c59-4366-b90f-038b7708ed40\") " Jul 21 18:01:59.154091 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.154061 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-console-config" (OuterVolumeSpecName: "console-config") pod "8eb60fee-4c59-4366-b90f-038b7708ed40" (UID: "8eb60fee-4c59-4366-b90f-038b7708ed40"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:01:59.154182 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.154112 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8eb60fee-4c59-4366-b90f-038b7708ed40" (UID: "8eb60fee-4c59-4366-b90f-038b7708ed40"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:01:59.154182 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.154123 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-service-ca" (OuterVolumeSpecName: "service-ca") pod "8eb60fee-4c59-4366-b90f-038b7708ed40" (UID: "8eb60fee-4c59-4366-b90f-038b7708ed40"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:01:59.155831 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.155807 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8eb60fee-4c59-4366-b90f-038b7708ed40-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8eb60fee-4c59-4366-b90f-038b7708ed40" (UID: "8eb60fee-4c59-4366-b90f-038b7708ed40"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:01:59.155905 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.155880 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eb60fee-4c59-4366-b90f-038b7708ed40-kube-api-access-p6w2s" (OuterVolumeSpecName: "kube-api-access-p6w2s") pod "8eb60fee-4c59-4366-b90f-038b7708ed40" (UID: "8eb60fee-4c59-4366-b90f-038b7708ed40"). InnerVolumeSpecName "kube-api-access-p6w2s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:01:59.155974 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.155953 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8eb60fee-4c59-4366-b90f-038b7708ed40-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8eb60fee-4c59-4366-b90f-038b7708ed40" (UID: "8eb60fee-4c59-4366-b90f-038b7708ed40"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:01:59.254657 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.254622 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6w2s\" (UniqueName: \"kubernetes.io/projected/8eb60fee-4c59-4366-b90f-038b7708ed40-kube-api-access-p6w2s\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:01:59.254657 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.254654 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-oauth-serving-cert\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:01:59.254657 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.254663 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-console-config\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:01:59.254891 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.254673 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8eb60fee-4c59-4366-b90f-038b7708ed40-console-oauth-config\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:01:59.254891 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.254681 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8eb60fee-4c59-4366-b90f-038b7708ed40-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:01:59.254891 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.254690 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8eb60fee-4c59-4366-b90f-038b7708ed40-console-serving-cert\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:01:59.497891 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.497843 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6774fd47f-flh6l" podUID="8586960d-dd9a-45be-89e0-db7ab33d749c" containerName="registry" containerID="cri-o://f8b76fe184462e859c7b1a5a40a879d812d2ccaaff69eca02b995506540d107a" gracePeriod=30 Jul 21 18:01:59.772060 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.771976 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:01:59.842728 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.842699 2568 generic.go:356] "Generic (PLEG): container finished" podID="8586960d-dd9a-45be-89e0-db7ab33d749c" containerID="f8b76fe184462e859c7b1a5a40a879d812d2ccaaff69eca02b995506540d107a" exitCode=0 Jul 21 18:01:59.842905 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.842743 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6774fd47f-flh6l" Jul 21 18:01:59.842905 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.842774 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6774fd47f-flh6l" event={"ID":"8586960d-dd9a-45be-89e0-db7ab33d749c","Type":"ContainerDied","Data":"f8b76fe184462e859c7b1a5a40a879d812d2ccaaff69eca02b995506540d107a"} Jul 21 18:01:59.842905 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.842863 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6774fd47f-flh6l" event={"ID":"8586960d-dd9a-45be-89e0-db7ab33d749c","Type":"ContainerDied","Data":"6c0d71dcbf22f1265554bc9508937f41894bafbc150bbabfc0dadbc785d2b852"} Jul 21 18:01:59.842905 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.842895 2568 scope.go:117] "RemoveContainer" containerID="f8b76fe184462e859c7b1a5a40a879d812d2ccaaff69eca02b995506540d107a" Jul 21 18:01:59.844220 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.844111 2568 generic.go:356] "Generic (PLEG): container finished" podID="8eb60fee-4c59-4366-b90f-038b7708ed40" containerID="525bff3c0984153ee9fb687f662632b864a7b0d5e092922352fa49a682d1f223" exitCode=0 Jul 21 18:01:59.844220 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.844168 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85bb7857d-2t948" Jul 21 18:01:59.844220 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.844175 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85bb7857d-2t948" event={"ID":"8eb60fee-4c59-4366-b90f-038b7708ed40","Type":"ContainerDied","Data":"525bff3c0984153ee9fb687f662632b864a7b0d5e092922352fa49a682d1f223"} Jul 21 18:01:59.844220 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.844212 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85bb7857d-2t948" event={"ID":"8eb60fee-4c59-4366-b90f-038b7708ed40","Type":"ContainerDied","Data":"2414b8741658e24781b787a33545671cabef31836d7a7e7bddea8ac38f1a587e"} Jul 21 18:01:59.860070 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.860041 2568 scope.go:117] "RemoveContainer" containerID="f8b76fe184462e859c7b1a5a40a879d812d2ccaaff69eca02b995506540d107a" Jul 21 18:01:59.860423 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:01:59.860391 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8b76fe184462e859c7b1a5a40a879d812d2ccaaff69eca02b995506540d107a\": container with ID starting with f8b76fe184462e859c7b1a5a40a879d812d2ccaaff69eca02b995506540d107a not found: ID does not exist" containerID="f8b76fe184462e859c7b1a5a40a879d812d2ccaaff69eca02b995506540d107a" Jul 21 18:01:59.860522 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.860433 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8b76fe184462e859c7b1a5a40a879d812d2ccaaff69eca02b995506540d107a"} err="failed to get container status \"f8b76fe184462e859c7b1a5a40a879d812d2ccaaff69eca02b995506540d107a\": rpc error: code = NotFound desc = could not find container \"f8b76fe184462e859c7b1a5a40a879d812d2ccaaff69eca02b995506540d107a\": container with ID starting with f8b76fe184462e859c7b1a5a40a879d812d2ccaaff69eca02b995506540d107a not found: ID does not exist" Jul 21 18:01:59.860522 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.860465 2568 scope.go:117] "RemoveContainer" containerID="525bff3c0984153ee9fb687f662632b864a7b0d5e092922352fa49a682d1f223" Jul 21 18:01:59.860629 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.860609 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85bb7857d-2t948"] Jul 21 18:01:59.864627 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.864606 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85bb7857d-2t948"] Jul 21 18:01:59.868277 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.868261 2568 scope.go:117] "RemoveContainer" containerID="525bff3c0984153ee9fb687f662632b864a7b0d5e092922352fa49a682d1f223" Jul 21 18:01:59.868548 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:01:59.868531 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"525bff3c0984153ee9fb687f662632b864a7b0d5e092922352fa49a682d1f223\": container with ID starting with 525bff3c0984153ee9fb687f662632b864a7b0d5e092922352fa49a682d1f223 not found: ID does not exist" containerID="525bff3c0984153ee9fb687f662632b864a7b0d5e092922352fa49a682d1f223" Jul 21 18:01:59.868592 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.868555 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"525bff3c0984153ee9fb687f662632b864a7b0d5e092922352fa49a682d1f223"} err="failed to get container status \"525bff3c0984153ee9fb687f662632b864a7b0d5e092922352fa49a682d1f223\": rpc error: code = NotFound desc = could not find container \"525bff3c0984153ee9fb687f662632b864a7b0d5e092922352fa49a682d1f223\": container with ID starting with 525bff3c0984153ee9fb687f662632b864a7b0d5e092922352fa49a682d1f223 not found: ID does not exist" Jul 21 18:01:59.960904 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.960866 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8586960d-dd9a-45be-89e0-db7ab33d749c-ca-trust-extracted\") pod \"8586960d-dd9a-45be-89e0-db7ab33d749c\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " Jul 21 18:01:59.960904 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.960909 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8586960d-dd9a-45be-89e0-db7ab33d749c-trusted-ca\") pod \"8586960d-dd9a-45be-89e0-db7ab33d749c\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " Jul 21 18:01:59.961141 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.960959 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8586960d-dd9a-45be-89e0-db7ab33d749c-image-registry-private-configuration\") pod \"8586960d-dd9a-45be-89e0-db7ab33d749c\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " Jul 21 18:01:59.961141 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.960994 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-bound-sa-token\") pod \"8586960d-dd9a-45be-89e0-db7ab33d749c\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " Jul 21 18:01:59.961141 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.961026 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7lfv\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-kube-api-access-g7lfv\") pod \"8586960d-dd9a-45be-89e0-db7ab33d749c\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " Jul 21 18:01:59.961141 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.961056 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8586960d-dd9a-45be-89e0-db7ab33d749c-installation-pull-secrets\") pod \"8586960d-dd9a-45be-89e0-db7ab33d749c\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " Jul 21 18:01:59.961141 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.961089 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls\") pod \"8586960d-dd9a-45be-89e0-db7ab33d749c\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " Jul 21 18:01:59.961141 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.961143 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-certificates\") pod \"8586960d-dd9a-45be-89e0-db7ab33d749c\" (UID: \"8586960d-dd9a-45be-89e0-db7ab33d749c\") " Jul 21 18:01:59.961710 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.961679 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8586960d-dd9a-45be-89e0-db7ab33d749c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8586960d-dd9a-45be-89e0-db7ab33d749c" (UID: "8586960d-dd9a-45be-89e0-db7ab33d749c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:01:59.961888 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.961692 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8586960d-dd9a-45be-89e0-db7ab33d749c" (UID: "8586960d-dd9a-45be-89e0-db7ab33d749c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:01:59.963759 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.963726 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8586960d-dd9a-45be-89e0-db7ab33d749c" (UID: "8586960d-dd9a-45be-89e0-db7ab33d749c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:01:59.963874 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.963757 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8586960d-dd9a-45be-89e0-db7ab33d749c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8586960d-dd9a-45be-89e0-db7ab33d749c" (UID: "8586960d-dd9a-45be-89e0-db7ab33d749c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:01:59.963874 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.963803 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-kube-api-access-g7lfv" (OuterVolumeSpecName: "kube-api-access-g7lfv") pod "8586960d-dd9a-45be-89e0-db7ab33d749c" (UID: "8586960d-dd9a-45be-89e0-db7ab33d749c"). InnerVolumeSpecName "kube-api-access-g7lfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:01:59.963874 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.963806 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8586960d-dd9a-45be-89e0-db7ab33d749c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8586960d-dd9a-45be-89e0-db7ab33d749c" (UID: "8586960d-dd9a-45be-89e0-db7ab33d749c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:01:59.964068 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.963883 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8586960d-dd9a-45be-89e0-db7ab33d749c" (UID: "8586960d-dd9a-45be-89e0-db7ab33d749c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:01:59.970336 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:01:59.970310 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8586960d-dd9a-45be-89e0-db7ab33d749c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8586960d-dd9a-45be-89e0-db7ab33d749c" (UID: "8586960d-dd9a-45be-89e0-db7ab33d749c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:02:00.062298 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:00.062209 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8586960d-dd9a-45be-89e0-db7ab33d749c-image-registry-private-configuration\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:02:00.062298 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:00.062241 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-bound-sa-token\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:02:00.062298 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:00.062254 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7lfv\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-kube-api-access-g7lfv\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:02:00.062298 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:00.062267 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8586960d-dd9a-45be-89e0-db7ab33d749c-installation-pull-secrets\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:02:00.062298 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:00.062275 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-tls\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:02:00.062298 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:00.062285 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8586960d-dd9a-45be-89e0-db7ab33d749c-registry-certificates\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:02:00.062298 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:00.062293 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8586960d-dd9a-45be-89e0-db7ab33d749c-ca-trust-extracted\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:02:00.062298 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:00.062302 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8586960d-dd9a-45be-89e0-db7ab33d749c-trusted-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:02:00.164956 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:00.164901 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6774fd47f-flh6l"] Jul 21 18:02:00.169330 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:00.169297 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6774fd47f-flh6l"] Jul 21 18:02:01.218187 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:01.218142 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8586960d-dd9a-45be-89e0-db7ab33d749c" path="/var/lib/kubelet/pods/8586960d-dd9a-45be-89e0-db7ab33d749c/volumes" Jul 21 18:02:01.218697 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:01.218676 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eb60fee-4c59-4366-b90f-038b7708ed40" path="/var/lib/kubelet/pods/8eb60fee-4c59-4366-b90f-038b7708ed40/volumes" Jul 21 18:02:01.422201 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:01.422171 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:02:01.826823 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:01.826782 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:02:01.855843 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:01.855805 2568 generic.go:356] "Generic (PLEG): container finished" podID="cf634bfc-df89-4e85-874a-33bd28e7eb68" containerID="d0527652945e077ffcbb7a0c9a0835aa03e584484b3d2099cea4cdc8a4dcb6c9" exitCode=0 Jul 21 18:02:01.856022 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:01.855883 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" event={"ID":"cf634bfc-df89-4e85-874a-33bd28e7eb68","Type":"ContainerDied","Data":"d0527652945e077ffcbb7a0c9a0835aa03e584484b3d2099cea4cdc8a4dcb6c9"} Jul 21 18:02:01.856232 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:01.856218 2568 scope.go:117] "RemoveContainer" containerID="d0527652945e077ffcbb7a0c9a0835aa03e584484b3d2099cea4cdc8a4dcb6c9" Jul 21 18:02:02.862020 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:02.861982 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-llddq" event={"ID":"cf634bfc-df89-4e85-874a-33bd28e7eb68","Type":"ContainerStarted","Data":"40ef35ba465b7d054312a0e3ebec30225927b317c12668710f87ed83aabcebc6"} Jul 21 18:02:03.716523 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:03.716489 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:02:03.850615 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:03.850573 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:02:03.926203 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:03.926168 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:02:05.959841 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:05.959808 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:02:07.883256 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:07.883224 2568 generic.go:356] "Generic (PLEG): container finished" podID="80cad807-9199-42c6-8b46-8d6887025525" containerID="b08192da38338f3f40b068cb976180c93d5a920c010899b6be4e23a2cdcbc1fb" exitCode=0 Jul 21 18:02:07.883615 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:07.883309 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ztxpr" event={"ID":"80cad807-9199-42c6-8b46-8d6887025525","Type":"ContainerDied","Data":"b08192da38338f3f40b068cb976180c93d5a920c010899b6be4e23a2cdcbc1fb"} Jul 21 18:02:07.883672 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:07.883643 2568 scope.go:117] "RemoveContainer" containerID="b08192da38338f3f40b068cb976180c93d5a920c010899b6be4e23a2cdcbc1fb" Jul 21 18:02:08.889271 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:08.889238 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ztxpr" event={"ID":"80cad807-9199-42c6-8b46-8d6887025525","Type":"ContainerStarted","Data":"a226b124d6f6ba460e6e5f973781905dbaee022fb0e8596f387a5fb986d9d851"} Jul 21 18:02:11.601431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:11.601393 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:02:11.904910 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:11.904809 2568 generic.go:356] "Generic (PLEG): container finished" podID="46b0651b-2ded-4eee-b247-cf1404ba7ba3" containerID="4e397a4157fc19e7ba84f4c99e9ffc05fad4ac5c4884b207f05917a0d290bbff" exitCode=0 Jul 21 18:02:11.905080 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:11.904899 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" event={"ID":"46b0651b-2ded-4eee-b247-cf1404ba7ba3","Type":"ContainerDied","Data":"4e397a4157fc19e7ba84f4c99e9ffc05fad4ac5c4884b207f05917a0d290bbff"} Jul 21 18:02:11.905250 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:11.905235 2568 scope.go:117] "RemoveContainer" containerID="4e397a4157fc19e7ba84f4c99e9ffc05fad4ac5c4884b207f05917a0d290bbff" Jul 21 18:02:12.643714 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:12.643678 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:02:12.910299 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:12.910201 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-tcd2b" event={"ID":"46b0651b-2ded-4eee-b247-cf1404ba7ba3","Type":"ContainerStarted","Data":"7e98f98a5d09e4f0abdf8570d5428dd719d9a5245ed37fae86f79aef20649766"} Jul 21 18:02:41.726185 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:41.726144 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:02:41.803209 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:41.803173 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:02:52.841105 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:52.841007 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:02:53.052420 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:53.052372 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:02:57.949566 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.949531 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-77544fd5d-qszwk"] Jul 21 18:02:57.949973 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.949823 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8586960d-dd9a-45be-89e0-db7ab33d749c" containerName="registry" Jul 21 18:02:57.949973 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.949834 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8586960d-dd9a-45be-89e0-db7ab33d749c" containerName="registry" Jul 21 18:02:57.949973 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.949853 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eb60fee-4c59-4366-b90f-038b7708ed40" containerName="console" Jul 21 18:02:57.949973 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.949861 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eb60fee-4c59-4366-b90f-038b7708ed40" containerName="console" Jul 21 18:02:57.949973 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.949949 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8586960d-dd9a-45be-89e0-db7ab33d749c" containerName="registry" Jul 21 18:02:57.949973 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.949959 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eb60fee-4c59-4366-b90f-038b7708ed40" containerName="console" Jul 21 18:02:57.952991 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.952970 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:57.957117 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.957092 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 21 18:02:57.957251 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.957092 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 21 18:02:57.957251 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.957166 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-72sgl\"" Jul 21 18:02:57.957355 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.957339 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 21 18:02:57.957410 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.957388 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 21 18:02:57.957600 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.957587 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 21 18:02:57.962763 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.962733 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 21 18:02:57.974044 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:57.974013 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77544fd5d-qszwk"] Jul 21 18:02:58.090871 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.090831 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6eac6911-3847-4b9d-ad58-220b04d1fa9d-telemeter-client-tls\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.091061 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.090875 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eac6911-3847-4b9d-ad58-220b04d1fa9d-serving-certs-ca-bundle\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.091061 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.090904 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6eac6911-3847-4b9d-ad58-220b04d1fa9d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.091061 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.091010 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6eac6911-3847-4b9d-ad58-220b04d1fa9d-metrics-client-ca\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.091061 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.091047 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6eac6911-3847-4b9d-ad58-220b04d1fa9d-federate-client-tls\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.091197 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.091117 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6eac6911-3847-4b9d-ad58-220b04d1fa9d-secret-telemeter-client\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.091197 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.091142 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eac6911-3847-4b9d-ad58-220b04d1fa9d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.091197 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.091165 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsp82\" (UniqueName: \"kubernetes.io/projected/6eac6911-3847-4b9d-ad58-220b04d1fa9d-kube-api-access-lsp82\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.192172 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.192131 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eac6911-3847-4b9d-ad58-220b04d1fa9d-serving-certs-ca-bundle\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.192172 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.192171 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6eac6911-3847-4b9d-ad58-220b04d1fa9d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.192464 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.192196 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6eac6911-3847-4b9d-ad58-220b04d1fa9d-metrics-client-ca\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.192464 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.192217 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6eac6911-3847-4b9d-ad58-220b04d1fa9d-federate-client-tls\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.192464 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.192246 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6eac6911-3847-4b9d-ad58-220b04d1fa9d-secret-telemeter-client\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.192464 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.192263 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eac6911-3847-4b9d-ad58-220b04d1fa9d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.192464 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.192280 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsp82\" (UniqueName: \"kubernetes.io/projected/6eac6911-3847-4b9d-ad58-220b04d1fa9d-kube-api-access-lsp82\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.192464 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.192311 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6eac6911-3847-4b9d-ad58-220b04d1fa9d-telemeter-client-tls\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.193032 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.193002 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eac6911-3847-4b9d-ad58-220b04d1fa9d-serving-certs-ca-bundle\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.193148 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.193084 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6eac6911-3847-4b9d-ad58-220b04d1fa9d-metrics-client-ca\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.193377 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.193352 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eac6911-3847-4b9d-ad58-220b04d1fa9d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.195044 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.195019 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6eac6911-3847-4b9d-ad58-220b04d1fa9d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.195308 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.195289 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6eac6911-3847-4b9d-ad58-220b04d1fa9d-telemeter-client-tls\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.195474 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.195447 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6eac6911-3847-4b9d-ad58-220b04d1fa9d-secret-telemeter-client\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.195660 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.195637 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6eac6911-3847-4b9d-ad58-220b04d1fa9d-federate-client-tls\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.201602 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.201541 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsp82\" (UniqueName: \"kubernetes.io/projected/6eac6911-3847-4b9d-ad58-220b04d1fa9d-kube-api-access-lsp82\") pod \"telemeter-client-77544fd5d-qszwk\" (UID: \"6eac6911-3847-4b9d-ad58-220b04d1fa9d\") " pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.262910 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.262864 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" Jul 21 18:02:58.403745 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:58.403706 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77544fd5d-qszwk"] Jul 21 18:02:58.407545 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:02:58.407522 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eac6911_3847_4b9d_ad58_220b04d1fa9d.slice/crio-a9db72dc68ebd0da850bfef2f74e8a2be7c1900a5eab0d57fbd0907a9f451479 WatchSource:0}: Error finding container a9db72dc68ebd0da850bfef2f74e8a2be7c1900a5eab0d57fbd0907a9f451479: Status 404 returned error can't find the container with id a9db72dc68ebd0da850bfef2f74e8a2be7c1900a5eab0d57fbd0907a9f451479 Jul 21 18:02:59.090555 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:02:59.090521 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" event={"ID":"6eac6911-3847-4b9d-ad58-220b04d1fa9d","Type":"ContainerStarted","Data":"a9db72dc68ebd0da850bfef2f74e8a2be7c1900a5eab0d57fbd0907a9f451479"} Jul 21 18:03:01.100149 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:01.100113 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" event={"ID":"6eac6911-3847-4b9d-ad58-220b04d1fa9d","Type":"ContainerStarted","Data":"6dd51f48ed294597e818bdb0dc7782cab20ad101fb0c4f42b90bc176c8b3db1e"} Jul 21 18:03:02.105977 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:02.105925 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" event={"ID":"6eac6911-3847-4b9d-ad58-220b04d1fa9d","Type":"ContainerStarted","Data":"dd1bf3f05d8df82bff68a92b2f15f21fe880ac59ebf863c150df06ed64cbe1db"} Jul 21 18:03:02.106373 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:02.106004 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" event={"ID":"6eac6911-3847-4b9d-ad58-220b04d1fa9d","Type":"ContainerStarted","Data":"fcf104932934abfd3d6c4728f9b7c63a73dbc1dfb8e463cf8d306ef4bac858ab"} Jul 21 18:03:02.132605 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:02.132549 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-77544fd5d-qszwk" podStartSLOduration=1.845971015 podStartE2EDuration="5.132527203s" totalImagesPullingTime="3.286556188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:02:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:03:02.132020701 +0000 UTC m=+183.400332144" watchObservedRunningTime="2026-07-21 18:03:02.132527203 +0000 UTC m=+183.400838647" Jul 21 18:03:03.047321 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.047284 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55799bb85d-546mv"] Jul 21 18:03:03.050706 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.050675 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.054281 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.054255 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 21 18:03:03.055532 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.055498 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hntdr\"" Jul 21 18:03:03.055626 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.055598 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 21 18:03:03.055694 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.055661 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 21 18:03:03.055849 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.055829 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 21 18:03:03.056010 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.055858 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 21 18:03:03.061470 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.061447 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 21 18:03:03.062045 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.062023 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55799bb85d-546mv"] Jul 21 18:03:03.129564 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.129527 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:03.134281 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.134260 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-trusted-ca-bundle\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.134343 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.134308 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-serving-cert\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.134392 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.134374 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-config\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.134427 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.134413 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tv5h\" (UniqueName: \"kubernetes.io/projected/a42066f9-434e-4d50-8cc6-39cb66ab6950-kube-api-access-5tv5h\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.134462 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.134447 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-oauth-config\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.134494 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.134462 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-service-ca\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.134494 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.134479 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-oauth-serving-cert\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.235479 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.235443 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-oauth-config\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.235479 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.235480 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-service-ca\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.235739 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.235504 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-oauth-serving-cert\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.235739 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.235531 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-trusted-ca-bundle\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.235914 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.235885 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-serving-cert\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.236106 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.236077 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-config\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.236182 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.236133 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tv5h\" (UniqueName: \"kubernetes.io/projected/a42066f9-434e-4d50-8cc6-39cb66ab6950-kube-api-access-5tv5h\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.236278 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.236257 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-oauth-serving-cert\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.236334 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.236315 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-service-ca\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.236457 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.236427 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-trusted-ca-bundle\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.236789 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.236769 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-config\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.238127 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.238106 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-oauth-config\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.238411 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.238392 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-serving-cert\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.256075 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.256046 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tv5h\" (UniqueName: \"kubernetes.io/projected/a42066f9-434e-4d50-8cc6-39cb66ab6950-kube-api-access-5tv5h\") pod \"console-55799bb85d-546mv\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.266007 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.265985 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:03.362570 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.362530 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:03.502282 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:03.502252 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55799bb85d-546mv"] Jul 21 18:03:03.505684 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:03:03.505654 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda42066f9_434e_4d50_8cc6_39cb66ab6950.slice/crio-8755779ce62fe474aa9c26b9d183aff1cfba67a721bfa138cfd8dd908a8973f3 WatchSource:0}: Error finding container 8755779ce62fe474aa9c26b9d183aff1cfba67a721bfa138cfd8dd908a8973f3: Status 404 returned error can't find the container with id 8755779ce62fe474aa9c26b9d183aff1cfba67a721bfa138cfd8dd908a8973f3 Jul 21 18:03:04.115831 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:04.115785 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55799bb85d-546mv" event={"ID":"a42066f9-434e-4d50-8cc6-39cb66ab6950","Type":"ContainerStarted","Data":"9d6e33c7e68347a591e9bb1dcb47509c805e888da1be5fbdfb7991386e31ea48"} Jul 21 18:03:04.115831 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:04.115834 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55799bb85d-546mv" event={"ID":"a42066f9-434e-4d50-8cc6-39cb66ab6950","Type":"ContainerStarted","Data":"8755779ce62fe474aa9c26b9d183aff1cfba67a721bfa138cfd8dd908a8973f3"} Jul 21 18:03:04.137085 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:04.137023 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55799bb85d-546mv" podStartSLOduration=1.13700565 podStartE2EDuration="1.13700565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:03:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:03:04.135686772 +0000 UTC m=+185.403998213" watchObservedRunningTime="2026-07-21 18:03:04.13700565 +0000 UTC m=+185.405317093" Jul 21 18:03:08.145105 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:08.145069 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:08.253309 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:08.253277 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:13.362817 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:13.362775 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:13.363326 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:13.362836 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:13.367647 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:13.367625 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:14.160052 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.160020 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:14.293368 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.293331 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d8bb9f966-tcgsr"] Jul 21 18:03:14.299491 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.299452 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.310811 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.310785 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d8bb9f966-tcgsr"] Jul 21 18:03:14.357809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.357751 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:14.431151 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.431044 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-console-oauth-config\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.431151 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.431145 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-service-ca\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.431599 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.431185 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-oauth-serving-cert\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.431599 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.431226 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-console-config\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.431599 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.431255 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdcnp\" (UniqueName: \"kubernetes.io/projected/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-kube-api-access-bdcnp\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.431599 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.431320 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-console-serving-cert\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.431599 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.431404 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-trusted-ca-bundle\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.432467 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.432444 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:14.532853 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.532804 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-service-ca\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.532853 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.532860 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-oauth-serving-cert\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.533132 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.532977 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-console-config\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.533132 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.533020 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdcnp\" (UniqueName: \"kubernetes.io/projected/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-kube-api-access-bdcnp\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.533132 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.533045 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-console-serving-cert\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.533132 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.533061 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-trusted-ca-bundle\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.533132 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.533130 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-console-oauth-config\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.533712 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.533685 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-service-ca\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.533843 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.533731 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-oauth-serving-cert\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.533843 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.533749 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-console-config\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.533978 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.533880 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-trusted-ca-bundle\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.535447 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.535419 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-console-serving-cert\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.535542 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.535493 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-console-oauth-config\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.542180 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.542156 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdcnp\" (UniqueName: \"kubernetes.io/projected/6921d7b6-2fc7-46e1-8a34-b544fa62e23d-kube-api-access-bdcnp\") pod \"console-d8bb9f966-tcgsr\" (UID: \"6921d7b6-2fc7-46e1-8a34-b544fa62e23d\") " pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.609268 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.609233 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:14.666913 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.666872 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:14.742067 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.742043 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d8bb9f966-tcgsr"] Jul 21 18:03:14.744201 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:03:14.744162 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6921d7b6_2fc7_46e1_8a34_b544fa62e23d.slice/crio-85dde8faa8e361c63370bdcc612eec5eba6002dddba3f28070829087361f3e92 WatchSource:0}: Error finding container 85dde8faa8e361c63370bdcc612eec5eba6002dddba3f28070829087361f3e92: Status 404 returned error can't find the container with id 85dde8faa8e361c63370bdcc612eec5eba6002dddba3f28070829087361f3e92 Jul 21 18:03:14.750915 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:14.750889 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:15.161483 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:15.161441 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d8bb9f966-tcgsr" event={"ID":"6921d7b6-2fc7-46e1-8a34-b544fa62e23d","Type":"ContainerStarted","Data":"9bd2caa878ff7977f03155499fa7ca2f1ce28a0bc75384ec8933220605e4d02c"} Jul 21 18:03:15.161654 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:15.161505 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d8bb9f966-tcgsr" event={"ID":"6921d7b6-2fc7-46e1-8a34-b544fa62e23d","Type":"ContainerStarted","Data":"85dde8faa8e361c63370bdcc612eec5eba6002dddba3f28070829087361f3e92"} Jul 21 18:03:15.183359 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:15.183309 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d8bb9f966-tcgsr" podStartSLOduration=1.183293538 podStartE2EDuration="1.183293538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:03:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:03:15.181708829 +0000 UTC m=+196.450020295" watchObservedRunningTime="2026-07-21 18:03:15.183293538 +0000 UTC m=+196.451604980" Jul 21 18:03:20.115336 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:20.115301 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:20.178722 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:20.178687 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:24.610325 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:24.610285 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:24.610720 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:24.610384 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:24.615129 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:24.615106 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:25.204112 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:25.204078 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d8bb9f966-tcgsr" Jul 21 18:03:25.257011 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:25.256977 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55799bb85d-546mv"] Jul 21 18:03:25.378425 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:25.378394 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:25.481483 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:25.481401 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:25.578407 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:25.578376 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:26.610400 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:26.610361 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:26.710420 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:26.710388 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:26.791674 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:26.791626 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:38.911152 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:38.911107 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:38.991737 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:38.991697 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:03:50.283391 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.283341 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55799bb85d-546mv" podUID="a42066f9-434e-4d50-8cc6-39cb66ab6950" containerName="console" containerID="cri-o://9d6e33c7e68347a591e9bb1dcb47509c805e888da1be5fbdfb7991386e31ea48" gracePeriod=15 Jul 21 18:03:50.524553 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.524524 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:50.627809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.627769 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-serving-cert\") pod \"a42066f9-434e-4d50-8cc6-39cb66ab6950\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " Jul 21 18:03:50.627809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.627809 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-oauth-serving-cert\") pod \"a42066f9-434e-4d50-8cc6-39cb66ab6950\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " Jul 21 18:03:50.628073 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.627866 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-service-ca\") pod \"a42066f9-434e-4d50-8cc6-39cb66ab6950\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " Jul 21 18:03:50.628073 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.627924 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-oauth-config\") pod \"a42066f9-434e-4d50-8cc6-39cb66ab6950\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " Jul 21 18:03:50.628073 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.628007 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-trusted-ca-bundle\") pod \"a42066f9-434e-4d50-8cc6-39cb66ab6950\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " Jul 21 18:03:50.628073 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.628030 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-config\") pod \"a42066f9-434e-4d50-8cc6-39cb66ab6950\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " Jul 21 18:03:50.628289 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.628092 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tv5h\" (UniqueName: \"kubernetes.io/projected/a42066f9-434e-4d50-8cc6-39cb66ab6950-kube-api-access-5tv5h\") pod \"a42066f9-434e-4d50-8cc6-39cb66ab6950\" (UID: \"a42066f9-434e-4d50-8cc6-39cb66ab6950\") " Jul 21 18:03:50.628344 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.628296 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a42066f9-434e-4d50-8cc6-39cb66ab6950" (UID: "a42066f9-434e-4d50-8cc6-39cb66ab6950"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:03:50.628436 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.628397 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-service-ca" (OuterVolumeSpecName: "service-ca") pod "a42066f9-434e-4d50-8cc6-39cb66ab6950" (UID: "a42066f9-434e-4d50-8cc6-39cb66ab6950"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:03:50.628493 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.628423 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a42066f9-434e-4d50-8cc6-39cb66ab6950" (UID: "a42066f9-434e-4d50-8cc6-39cb66ab6950"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:03:50.628681 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.628660 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-config" (OuterVolumeSpecName: "console-config") pod "a42066f9-434e-4d50-8cc6-39cb66ab6950" (UID: "a42066f9-434e-4d50-8cc6-39cb66ab6950"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:03:50.630165 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.630132 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a42066f9-434e-4d50-8cc6-39cb66ab6950" (UID: "a42066f9-434e-4d50-8cc6-39cb66ab6950"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:03:50.630165 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.630150 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a42066f9-434e-4d50-8cc6-39cb66ab6950" (UID: "a42066f9-434e-4d50-8cc6-39cb66ab6950"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:03:50.630332 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.630310 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a42066f9-434e-4d50-8cc6-39cb66ab6950-kube-api-access-5tv5h" (OuterVolumeSpecName: "kube-api-access-5tv5h") pod "a42066f9-434e-4d50-8cc6-39cb66ab6950" (UID: "a42066f9-434e-4d50-8cc6-39cb66ab6950"). InnerVolumeSpecName "kube-api-access-5tv5h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:03:50.729195 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.729145 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-trusted-ca-bundle\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:03:50.729195 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.729191 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-config\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:03:50.729195 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.729201 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tv5h\" (UniqueName: \"kubernetes.io/projected/a42066f9-434e-4d50-8cc6-39cb66ab6950-kube-api-access-5tv5h\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:03:50.729195 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.729212 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-serving-cert\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:03:50.729457 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.729222 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-oauth-serving-cert\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:03:50.729457 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.729230 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a42066f9-434e-4d50-8cc6-39cb66ab6950-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:03:50.729457 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:50.729239 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a42066f9-434e-4d50-8cc6-39cb66ab6950-console-oauth-config\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:03:51.300395 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:51.300356 2568 generic.go:356] "Generic (PLEG): container finished" podID="a42066f9-434e-4d50-8cc6-39cb66ab6950" containerID="9d6e33c7e68347a591e9bb1dcb47509c805e888da1be5fbdfb7991386e31ea48" exitCode=0 Jul 21 18:03:51.300867 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:51.300424 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55799bb85d-546mv" Jul 21 18:03:51.300867 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:51.300444 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55799bb85d-546mv" event={"ID":"a42066f9-434e-4d50-8cc6-39cb66ab6950","Type":"ContainerDied","Data":"9d6e33c7e68347a591e9bb1dcb47509c805e888da1be5fbdfb7991386e31ea48"} Jul 21 18:03:51.300867 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:51.300499 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55799bb85d-546mv" event={"ID":"a42066f9-434e-4d50-8cc6-39cb66ab6950","Type":"ContainerDied","Data":"8755779ce62fe474aa9c26b9d183aff1cfba67a721bfa138cfd8dd908a8973f3"} Jul 21 18:03:51.300867 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:51.300519 2568 scope.go:117] "RemoveContainer" containerID="9d6e33c7e68347a591e9bb1dcb47509c805e888da1be5fbdfb7991386e31ea48" Jul 21 18:03:51.308713 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:51.308693 2568 scope.go:117] "RemoveContainer" containerID="9d6e33c7e68347a591e9bb1dcb47509c805e888da1be5fbdfb7991386e31ea48" Jul 21 18:03:51.309056 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:03:51.309019 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d6e33c7e68347a591e9bb1dcb47509c805e888da1be5fbdfb7991386e31ea48\": container with ID starting with 9d6e33c7e68347a591e9bb1dcb47509c805e888da1be5fbdfb7991386e31ea48 not found: ID does not exist" containerID="9d6e33c7e68347a591e9bb1dcb47509c805e888da1be5fbdfb7991386e31ea48" Jul 21 18:03:51.309133 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:51.309065 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d6e33c7e68347a591e9bb1dcb47509c805e888da1be5fbdfb7991386e31ea48"} err="failed to get container status \"9d6e33c7e68347a591e9bb1dcb47509c805e888da1be5fbdfb7991386e31ea48\": rpc error: code = NotFound desc = could not find container \"9d6e33c7e68347a591e9bb1dcb47509c805e888da1be5fbdfb7991386e31ea48\": container with ID starting with 9d6e33c7e68347a591e9bb1dcb47509c805e888da1be5fbdfb7991386e31ea48 not found: ID does not exist" Jul 21 18:03:51.319422 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:51.319396 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55799bb85d-546mv"] Jul 21 18:03:51.323854 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:51.323835 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55799bb85d-546mv"] Jul 21 18:03:53.218123 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:03:53.218092 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a42066f9-434e-4d50-8cc6-39cb66ab6950" path="/var/lib/kubelet/pods/a42066f9-434e-4d50-8cc6-39cb66ab6950/volumes" Jul 21 18:04:04.916157 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:04.916119 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:04.994358 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:04.994306 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:14.144617 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:14.144576 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:14.212444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:14.212403 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:14.436479 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:14.436396 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:14.521306 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:14.521265 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:15.301724 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.301687 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5"] Jul 21 18:04:15.302096 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.302002 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a42066f9-434e-4d50-8cc6-39cb66ab6950" containerName="console" Jul 21 18:04:15.302096 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.302014 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a42066f9-434e-4d50-8cc6-39cb66ab6950" containerName="console" Jul 21 18:04:15.302096 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.302078 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a42066f9-434e-4d50-8cc6-39cb66ab6950" containerName="console" Jul 21 18:04:15.306431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.306413 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" Jul 21 18:04:15.310406 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.310382 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 21 18:04:15.310565 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.310541 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 21 18:04:15.311201 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.311183 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p47rs\"" Jul 21 18:04:15.317958 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.317922 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5"] Jul 21 18:04:15.337042 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.337016 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d49b\" (UniqueName: \"kubernetes.io/projected/7d225e5a-35ec-43fb-ac74-40f57cd03197-kube-api-access-5d49b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5\" (UID: \"7d225e5a-35ec-43fb-ac74-40f57cd03197\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" Jul 21 18:04:15.337145 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.337052 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d225e5a-35ec-43fb-ac74-40f57cd03197-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5\" (UID: \"7d225e5a-35ec-43fb-ac74-40f57cd03197\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" Jul 21 18:04:15.337145 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.337132 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d225e5a-35ec-43fb-ac74-40f57cd03197-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5\" (UID: \"7d225e5a-35ec-43fb-ac74-40f57cd03197\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" Jul 21 18:04:15.438374 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.438335 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d225e5a-35ec-43fb-ac74-40f57cd03197-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5\" (UID: \"7d225e5a-35ec-43fb-ac74-40f57cd03197\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" Jul 21 18:04:15.438554 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.438405 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d49b\" (UniqueName: \"kubernetes.io/projected/7d225e5a-35ec-43fb-ac74-40f57cd03197-kube-api-access-5d49b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5\" (UID: \"7d225e5a-35ec-43fb-ac74-40f57cd03197\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" Jul 21 18:04:15.438554 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.438445 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d225e5a-35ec-43fb-ac74-40f57cd03197-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5\" (UID: \"7d225e5a-35ec-43fb-ac74-40f57cd03197\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" Jul 21 18:04:15.438818 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.438799 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d225e5a-35ec-43fb-ac74-40f57cd03197-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5\" (UID: \"7d225e5a-35ec-43fb-ac74-40f57cd03197\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" Jul 21 18:04:15.438870 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.438810 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d225e5a-35ec-43fb-ac74-40f57cd03197-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5\" (UID: \"7d225e5a-35ec-43fb-ac74-40f57cd03197\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" Jul 21 18:04:15.447923 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.447899 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d49b\" (UniqueName: \"kubernetes.io/projected/7d225e5a-35ec-43fb-ac74-40f57cd03197-kube-api-access-5d49b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5\" (UID: \"7d225e5a-35ec-43fb-ac74-40f57cd03197\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" Jul 21 18:04:15.615408 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.615369 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" Jul 21 18:04:15.747760 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:15.747725 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5"] Jul 21 18:04:15.751770 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:04:15.751740 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d225e5a_35ec_43fb_ac74_40f57cd03197.slice/crio-239ea34c4990c3c6a8810d88a97463070bddcd0d55c9027f54d5f1acf01d790c WatchSource:0}: Error finding container 239ea34c4990c3c6a8810d88a97463070bddcd0d55c9027f54d5f1acf01d790c: Status 404 returned error can't find the container with id 239ea34c4990c3c6a8810d88a97463070bddcd0d55c9027f54d5f1acf01d790c Jul 21 18:04:16.397764 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:16.397726 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" event={"ID":"7d225e5a-35ec-43fb-ac74-40f57cd03197","Type":"ContainerStarted","Data":"239ea34c4990c3c6a8810d88a97463070bddcd0d55c9027f54d5f1acf01d790c"} Jul 21 18:04:23.432310 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:23.432264 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" event={"ID":"7d225e5a-35ec-43fb-ac74-40f57cd03197","Type":"ContainerStarted","Data":"f37d833b802107e8882cd1b2908ff70986796defdc9507c4774af25512ef93f9"} Jul 21 18:04:24.437921 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:24.437882 2568 generic.go:356] "Generic (PLEG): container finished" podID="7d225e5a-35ec-43fb-ac74-40f57cd03197" containerID="f37d833b802107e8882cd1b2908ff70986796defdc9507c4774af25512ef93f9" exitCode=0 Jul 21 18:04:24.438338 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:24.437976 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" event={"ID":"7d225e5a-35ec-43fb-ac74-40f57cd03197","Type":"ContainerDied","Data":"f37d833b802107e8882cd1b2908ff70986796defdc9507c4774af25512ef93f9"} Jul 21 18:04:27.451595 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:27.451558 2568 generic.go:356] "Generic (PLEG): container finished" podID="7d225e5a-35ec-43fb-ac74-40f57cd03197" containerID="e6829286dac48b9728fc027a771b3ced1b8f38abb6662baa83000948e9354bda" exitCode=0 Jul 21 18:04:27.451984 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:27.451640 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" event={"ID":"7d225e5a-35ec-43fb-ac74-40f57cd03197","Type":"ContainerDied","Data":"e6829286dac48b9728fc027a771b3ced1b8f38abb6662baa83000948e9354bda"} Jul 21 18:04:35.491013 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:35.490962 2568 generic.go:356] "Generic (PLEG): container finished" podID="7d225e5a-35ec-43fb-ac74-40f57cd03197" containerID="abc691dfd7e097e1aca86ca130f15f70356032fb7802e4bb1bef478f38d98740" exitCode=0 Jul 21 18:04:35.491377 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:35.491033 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" event={"ID":"7d225e5a-35ec-43fb-ac74-40f57cd03197","Type":"ContainerDied","Data":"abc691dfd7e097e1aca86ca130f15f70356032fb7802e4bb1bef478f38d98740"} Jul 21 18:04:36.618650 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:36.618620 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" Jul 21 18:04:36.722126 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:36.722093 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d225e5a-35ec-43fb-ac74-40f57cd03197-util\") pod \"7d225e5a-35ec-43fb-ac74-40f57cd03197\" (UID: \"7d225e5a-35ec-43fb-ac74-40f57cd03197\") " Jul 21 18:04:36.722126 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:36.722135 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d225e5a-35ec-43fb-ac74-40f57cd03197-bundle\") pod \"7d225e5a-35ec-43fb-ac74-40f57cd03197\" (UID: \"7d225e5a-35ec-43fb-ac74-40f57cd03197\") " Jul 21 18:04:36.722358 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:36.722228 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5d49b\" (UniqueName: \"kubernetes.io/projected/7d225e5a-35ec-43fb-ac74-40f57cd03197-kube-api-access-5d49b\") pod \"7d225e5a-35ec-43fb-ac74-40f57cd03197\" (UID: \"7d225e5a-35ec-43fb-ac74-40f57cd03197\") " Jul 21 18:04:36.723006 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:36.722973 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d225e5a-35ec-43fb-ac74-40f57cd03197-bundle" (OuterVolumeSpecName: "bundle") pod "7d225e5a-35ec-43fb-ac74-40f57cd03197" (UID: "7d225e5a-35ec-43fb-ac74-40f57cd03197"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:04:36.724441 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:36.724414 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d225e5a-35ec-43fb-ac74-40f57cd03197-kube-api-access-5d49b" (OuterVolumeSpecName: "kube-api-access-5d49b") pod "7d225e5a-35ec-43fb-ac74-40f57cd03197" (UID: "7d225e5a-35ec-43fb-ac74-40f57cd03197"). InnerVolumeSpecName "kube-api-access-5d49b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:04:36.726403 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:36.726380 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d225e5a-35ec-43fb-ac74-40f57cd03197-util" (OuterVolumeSpecName: "util") pod "7d225e5a-35ec-43fb-ac74-40f57cd03197" (UID: "7d225e5a-35ec-43fb-ac74-40f57cd03197"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:04:36.823064 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:36.822967 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5d49b\" (UniqueName: \"kubernetes.io/projected/7d225e5a-35ec-43fb-ac74-40f57cd03197-kube-api-access-5d49b\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:04:36.823064 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:36.823013 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d225e5a-35ec-43fb-ac74-40f57cd03197-util\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:04:36.823064 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:36.823026 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d225e5a-35ec-43fb-ac74-40f57cd03197-bundle\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:04:37.500501 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:37.500419 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" Jul 21 18:04:37.500501 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:37.500436 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xsbj5" event={"ID":"7d225e5a-35ec-43fb-ac74-40f57cd03197","Type":"ContainerDied","Data":"239ea34c4990c3c6a8810d88a97463070bddcd0d55c9027f54d5f1acf01d790c"} Jul 21 18:04:37.500501 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:37.500486 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="239ea34c4990c3c6a8810d88a97463070bddcd0d55c9027f54d5f1acf01d790c" Jul 21 18:04:41.432928 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:41.432897 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:42.294272 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:42.294237 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:43.505616 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.505582 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jszxz"] Jul 21 18:04:43.506024 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.505882 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d225e5a-35ec-43fb-ac74-40f57cd03197" containerName="pull" Jul 21 18:04:43.506024 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.505892 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d225e5a-35ec-43fb-ac74-40f57cd03197" containerName="pull" Jul 21 18:04:43.506024 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.505909 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d225e5a-35ec-43fb-ac74-40f57cd03197" containerName="util" Jul 21 18:04:43.506024 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.505915 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d225e5a-35ec-43fb-ac74-40f57cd03197" containerName="util" Jul 21 18:04:43.506024 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.505948 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d225e5a-35ec-43fb-ac74-40f57cd03197" containerName="extract" Jul 21 18:04:43.506024 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.505956 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d225e5a-35ec-43fb-ac74-40f57cd03197" containerName="extract" Jul 21 18:04:43.506024 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.506017 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d225e5a-35ec-43fb-ac74-40f57cd03197" containerName="extract" Jul 21 18:04:43.510330 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.510311 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jszxz" Jul 21 18:04:43.514210 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.514183 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-2v4pv\"" Jul 21 18:04:43.515531 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.515495 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 21 18:04:43.515729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.515554 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 21 18:04:43.531473 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.531441 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jszxz"] Jul 21 18:04:43.585103 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.585062 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/333ba91b-c30d-4d95-b06f-f04a8478c554-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jszxz\" (UID: \"333ba91b-c30d-4d95-b06f-f04a8478c554\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jszxz" Jul 21 18:04:43.585103 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.585104 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psvzp\" (UniqueName: \"kubernetes.io/projected/333ba91b-c30d-4d95-b06f-f04a8478c554-kube-api-access-psvzp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jszxz\" (UID: \"333ba91b-c30d-4d95-b06f-f04a8478c554\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jszxz" Jul 21 18:04:43.686307 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.686262 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/333ba91b-c30d-4d95-b06f-f04a8478c554-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jszxz\" (UID: \"333ba91b-c30d-4d95-b06f-f04a8478c554\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jszxz" Jul 21 18:04:43.686307 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.686305 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psvzp\" (UniqueName: \"kubernetes.io/projected/333ba91b-c30d-4d95-b06f-f04a8478c554-kube-api-access-psvzp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jszxz\" (UID: \"333ba91b-c30d-4d95-b06f-f04a8478c554\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jszxz" Jul 21 18:04:43.686713 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.686689 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/333ba91b-c30d-4d95-b06f-f04a8478c554-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jszxz\" (UID: \"333ba91b-c30d-4d95-b06f-f04a8478c554\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jszxz" Jul 21 18:04:43.695977 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.695951 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psvzp\" (UniqueName: \"kubernetes.io/projected/333ba91b-c30d-4d95-b06f-f04a8478c554-kube-api-access-psvzp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jszxz\" (UID: \"333ba91b-c30d-4d95-b06f-f04a8478c554\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jszxz" Jul 21 18:04:43.819630 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.819551 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jszxz" Jul 21 18:04:43.960522 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:43.960490 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jszxz"] Jul 21 18:04:43.963249 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:04:43.963220 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod333ba91b_c30d_4d95_b06f_f04a8478c554.slice/crio-ff9251a77bb2f6dcffd633cef76540770487754e919aa250c81cd669b908bf47 WatchSource:0}: Error finding container ff9251a77bb2f6dcffd633cef76540770487754e919aa250c81cd669b908bf47: Status 404 returned error can't find the container with id ff9251a77bb2f6dcffd633cef76540770487754e919aa250c81cd669b908bf47 Jul 21 18:04:44.528754 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:44.528716 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jszxz" event={"ID":"333ba91b-c30d-4d95-b06f-f04a8478c554","Type":"ContainerStarted","Data":"ff9251a77bb2f6dcffd633cef76540770487754e919aa250c81cd669b908bf47"} Jul 21 18:04:45.247878 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:45.247821 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:45.348244 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:45.348208 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:46.540532 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:46.540487 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jszxz" event={"ID":"333ba91b-c30d-4d95-b06f-f04a8478c554","Type":"ContainerStarted","Data":"19ef33637c18a6ded99367e2a91702259eefa74531137ac39d8ecebae375b2eb"} Jul 21 18:04:46.573413 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:46.573355 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jszxz" podStartSLOduration=1.725793838 podStartE2EDuration="3.57333739s" totalImagesPullingTime="1.847543552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:04:46.571880325 +0000 UTC m=+287.840191768" watchObservedRunningTime="2026-07-21 18:04:46.57333739 +0000 UTC m=+287.841648831" Jul 21 18:04:46.652774 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:46.652739 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:46.772434 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:46.772393 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:46.961664 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:46.961612 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:47.096496 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:47.096459 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:47.549427 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:47.549387 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:47.721001 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:47.720964 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:47.810697 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:47.810610 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684"] Jul 21 18:04:47.814247 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:47.814230 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" Jul 21 18:04:47.818103 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:47.818080 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 21 18:04:47.818103 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:47.818096 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p47rs\"" Jul 21 18:04:47.818276 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:47.818103 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 21 18:04:47.831445 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:47.831415 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684"] Jul 21 18:04:47.921154 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:47.921116 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684\" (UID: \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" Jul 21 18:04:47.921322 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:47.921179 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684\" (UID: \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" Jul 21 18:04:47.921322 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:47.921216 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66n4c\" (UniqueName: \"kubernetes.io/projected/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-kube-api-access-66n4c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684\" (UID: \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" Jul 21 18:04:48.022235 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:48.022194 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66n4c\" (UniqueName: \"kubernetes.io/projected/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-kube-api-access-66n4c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684\" (UID: \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" Jul 21 18:04:48.022411 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:48.022246 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684\" (UID: \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" Jul 21 18:04:48.022411 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:48.022295 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684\" (UID: \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" Jul 21 18:04:48.022632 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:48.022616 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684\" (UID: \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" Jul 21 18:04:48.022721 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:48.022698 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684\" (UID: \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" Jul 21 18:04:48.040396 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:48.040369 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66n4c\" (UniqueName: \"kubernetes.io/projected/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-kube-api-access-66n4c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684\" (UID: \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" Jul 21 18:04:48.123171 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:48.123127 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" Jul 21 18:04:48.259451 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:48.259426 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684"] Jul 21 18:04:48.262060 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:04:48.262030 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47f8d6e2_adb4_4a49_8da5_03939d9c4e69.slice/crio-24a41e9bbe808954530efcab4f2f4362339dfe2fa3cbd46ba1bff1005b98bcfa WatchSource:0}: Error finding container 24a41e9bbe808954530efcab4f2f4362339dfe2fa3cbd46ba1bff1005b98bcfa: Status 404 returned error can't find the container with id 24a41e9bbe808954530efcab4f2f4362339dfe2fa3cbd46ba1bff1005b98bcfa Jul 21 18:04:48.550046 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:48.549965 2568 generic.go:356] "Generic (PLEG): container finished" podID="47f8d6e2-adb4-4a49-8da5-03939d9c4e69" containerID="f77e98f4e864018b736190457ebc574d4cd1cab4b62a332e3888aa55f4b3e4be" exitCode=0 Jul 21 18:04:48.550449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:48.550073 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" event={"ID":"47f8d6e2-adb4-4a49-8da5-03939d9c4e69","Type":"ContainerDied","Data":"f77e98f4e864018b736190457ebc574d4cd1cab4b62a332e3888aa55f4b3e4be"} Jul 21 18:04:48.550449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:48.550109 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" event={"ID":"47f8d6e2-adb4-4a49-8da5-03939d9c4e69","Type":"ContainerStarted","Data":"24a41e9bbe808954530efcab4f2f4362339dfe2fa3cbd46ba1bff1005b98bcfa"} Jul 21 18:04:49.033749 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:49.033707 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:49.260544 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:49.260509 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:52.234391 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:52.234351 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:52.334669 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:52.334635 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:54.611371 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:54.611338 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wnwjt"] Jul 21 18:04:54.616066 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:54.616044 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wnwjt" Jul 21 18:04:54.620170 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:54.620146 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 21 18:04:54.620312 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:54.620276 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-z85tj\"" Jul 21 18:04:54.622333 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:54.622306 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 21 18:04:54.624407 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:54.624384 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wnwjt"] Jul 21 18:04:54.782277 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:54.782234 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4e644e0-cfde-46f1-b26a-4c7963a28ee6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wnwjt\" (UID: \"e4e644e0-cfde-46f1-b26a-4c7963a28ee6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wnwjt" Jul 21 18:04:54.782455 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:54.782296 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sljj\" (UniqueName: \"kubernetes.io/projected/e4e644e0-cfde-46f1-b26a-4c7963a28ee6-kube-api-access-7sljj\") pod \"cert-manager-webhook-6486ffd997-wnwjt\" (UID: \"e4e644e0-cfde-46f1-b26a-4c7963a28ee6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wnwjt" Jul 21 18:04:54.883750 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:54.883653 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4e644e0-cfde-46f1-b26a-4c7963a28ee6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wnwjt\" (UID: \"e4e644e0-cfde-46f1-b26a-4c7963a28ee6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wnwjt" Jul 21 18:04:54.883750 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:54.883712 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sljj\" (UniqueName: \"kubernetes.io/projected/e4e644e0-cfde-46f1-b26a-4c7963a28ee6-kube-api-access-7sljj\") pod \"cert-manager-webhook-6486ffd997-wnwjt\" (UID: \"e4e644e0-cfde-46f1-b26a-4c7963a28ee6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wnwjt" Jul 21 18:04:54.893700 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:54.893672 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4e644e0-cfde-46f1-b26a-4c7963a28ee6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wnwjt\" (UID: \"e4e644e0-cfde-46f1-b26a-4c7963a28ee6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wnwjt" Jul 21 18:04:54.893808 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:54.893752 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sljj\" (UniqueName: \"kubernetes.io/projected/e4e644e0-cfde-46f1-b26a-4c7963a28ee6-kube-api-access-7sljj\") pod \"cert-manager-webhook-6486ffd997-wnwjt\" (UID: \"e4e644e0-cfde-46f1-b26a-4c7963a28ee6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wnwjt" Jul 21 18:04:54.938896 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:54.938859 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wnwjt" Jul 21 18:04:55.078730 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:55.078695 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wnwjt"] Jul 21 18:04:55.082243 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:04:55.082220 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4e644e0_cfde_46f1_b26a_4c7963a28ee6.slice/crio-7427f4acfde95055d3e2045ad1db8f0358f06d3a4b7813450a26848533434605 WatchSource:0}: Error finding container 7427f4acfde95055d3e2045ad1db8f0358f06d3a4b7813450a26848533434605: Status 404 returned error can't find the container with id 7427f4acfde95055d3e2045ad1db8f0358f06d3a4b7813450a26848533434605 Jul 21 18:04:55.581483 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:55.581440 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wnwjt" event={"ID":"e4e644e0-cfde-46f1-b26a-4c7963a28ee6","Type":"ContainerStarted","Data":"7427f4acfde95055d3e2045ad1db8f0358f06d3a4b7813450a26848533434605"} Jul 21 18:04:56.256569 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:56.256531 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:04:58.037651 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.037568 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5qwbv"] Jul 21 18:04:58.040896 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.040877 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5qwbv" Jul 21 18:04:58.045397 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.045373 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-87wrf\"" Jul 21 18:04:58.054259 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.054234 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5qwbv"] Jul 21 18:04:58.215704 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.215666 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6g9w\" (UniqueName: \"kubernetes.io/projected/aa722eb2-c8f7-4233-881b-27d58a14f5b7-kube-api-access-b6g9w\") pod \"cert-manager-55b4d954d5-5qwbv\" (UID: \"aa722eb2-c8f7-4233-881b-27d58a14f5b7\") " pod="cert-manager/cert-manager-55b4d954d5-5qwbv" Jul 21 18:04:58.215876 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.215722 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa722eb2-c8f7-4233-881b-27d58a14f5b7-bound-sa-token\") pod \"cert-manager-55b4d954d5-5qwbv\" (UID: \"aa722eb2-c8f7-4233-881b-27d58a14f5b7\") " pod="cert-manager/cert-manager-55b4d954d5-5qwbv" Jul 21 18:04:58.316899 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.316809 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6g9w\" (UniqueName: \"kubernetes.io/projected/aa722eb2-c8f7-4233-881b-27d58a14f5b7-kube-api-access-b6g9w\") pod \"cert-manager-55b4d954d5-5qwbv\" (UID: \"aa722eb2-c8f7-4233-881b-27d58a14f5b7\") " pod="cert-manager/cert-manager-55b4d954d5-5qwbv" Jul 21 18:04:58.316899 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.316878 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa722eb2-c8f7-4233-881b-27d58a14f5b7-bound-sa-token\") pod \"cert-manager-55b4d954d5-5qwbv\" (UID: \"aa722eb2-c8f7-4233-881b-27d58a14f5b7\") " pod="cert-manager/cert-manager-55b4d954d5-5qwbv" Jul 21 18:04:58.325989 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.325957 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa722eb2-c8f7-4233-881b-27d58a14f5b7-bound-sa-token\") pod \"cert-manager-55b4d954d5-5qwbv\" (UID: \"aa722eb2-c8f7-4233-881b-27d58a14f5b7\") " pod="cert-manager/cert-manager-55b4d954d5-5qwbv" Jul 21 18:04:58.326277 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.326254 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6g9w\" (UniqueName: \"kubernetes.io/projected/aa722eb2-c8f7-4233-881b-27d58a14f5b7-kube-api-access-b6g9w\") pod \"cert-manager-55b4d954d5-5qwbv\" (UID: \"aa722eb2-c8f7-4233-881b-27d58a14f5b7\") " pod="cert-manager/cert-manager-55b4d954d5-5qwbv" Jul 21 18:04:58.349540 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.349513 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5qwbv" Jul 21 18:04:58.487961 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.487921 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5qwbv"] Jul 21 18:04:58.490274 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:04:58.490244 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa722eb2_c8f7_4233_881b_27d58a14f5b7.slice/crio-d0121ae93c106f2d606b93a73156a2432e1bf2e8268efed4a048ccd2956ff633 WatchSource:0}: Error finding container d0121ae93c106f2d606b93a73156a2432e1bf2e8268efed4a048ccd2956ff633: Status 404 returned error can't find the container with id d0121ae93c106f2d606b93a73156a2432e1bf2e8268efed4a048ccd2956ff633 Jul 21 18:04:58.597702 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.597667 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wnwjt" event={"ID":"e4e644e0-cfde-46f1-b26a-4c7963a28ee6","Type":"ContainerStarted","Data":"1c94f7fc7dc0c62802f4d3224f4627dbdb56d8026d594c4f03dc6aa9eecb3ed6"} Jul 21 18:04:58.597877 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.597721 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wnwjt" Jul 21 18:04:58.599037 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.599011 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5qwbv" event={"ID":"aa722eb2-c8f7-4233-881b-27d58a14f5b7","Type":"ContainerStarted","Data":"ec497f90ab9ba9203a26f3c671e01513bc1475ec5ea06136cd444527fde2d23f"} Jul 21 18:04:58.599120 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.599049 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5qwbv" event={"ID":"aa722eb2-c8f7-4233-881b-27d58a14f5b7","Type":"ContainerStarted","Data":"d0121ae93c106f2d606b93a73156a2432e1bf2e8268efed4a048ccd2956ff633"} Jul 21 18:04:58.616473 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.616425 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-wnwjt" podStartSLOduration=2.031724162 podStartE2EDuration="4.616410787s" totalImagesPullingTime="2.584686625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:04:58.615500095 +0000 UTC m=+299.883811553" watchObservedRunningTime="2026-07-21 18:04:58.616410787 +0000 UTC m=+299.884722230" Jul 21 18:04:58.634794 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:58.634743 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-5qwbv" podStartSLOduration=0.634727478 podStartE2EDuration="634.727478ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:04:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:04:58.632986018 +0000 UTC m=+299.901297465" watchObservedRunningTime="2026-07-21 18:04:58.634727478 +0000 UTC m=+299.903038919" Jul 21 18:04:59.082015 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:59.081918 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:04:59.083473 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:59.083454 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:04:59.088485 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:59.088463 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:04:59.090414 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:59.090391 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:04:59.093336 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:04:59.093318 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 21 18:05:04.605631 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:04.605601 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wnwjt" Jul 21 18:05:14.669288 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:14.669250 2568 generic.go:356] "Generic (PLEG): container finished" podID="47f8d6e2-adb4-4a49-8da5-03939d9c4e69" containerID="200368c1a65f3966fc15d2e1f90cce2db879bca752f5605d27108afa5d1a854a" exitCode=0 Jul 21 18:05:14.669707 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:14.669319 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" event={"ID":"47f8d6e2-adb4-4a49-8da5-03939d9c4e69","Type":"ContainerDied","Data":"200368c1a65f3966fc15d2e1f90cce2db879bca752f5605d27108afa5d1a854a"} Jul 21 18:05:15.675340 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:15.675306 2568 generic.go:356] "Generic (PLEG): container finished" podID="47f8d6e2-adb4-4a49-8da5-03939d9c4e69" containerID="055cc3c243dd14d69526c03b01988abc533f304f5c89e511c7461c405748f49f" exitCode=0 Jul 21 18:05:15.675729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:15.675357 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" event={"ID":"47f8d6e2-adb4-4a49-8da5-03939d9c4e69","Type":"ContainerDied","Data":"055cc3c243dd14d69526c03b01988abc533f304f5c89e511c7461c405748f49f"} Jul 21 18:05:16.806390 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:16.806364 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" Jul 21 18:05:16.876745 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:16.876700 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-bundle\") pod \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\" (UID: \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\") " Jul 21 18:05:16.876922 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:16.876759 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-util\") pod \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\" (UID: \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\") " Jul 21 18:05:16.876922 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:16.876805 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66n4c\" (UniqueName: \"kubernetes.io/projected/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-kube-api-access-66n4c\") pod \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\" (UID: \"47f8d6e2-adb4-4a49-8da5-03939d9c4e69\") " Jul 21 18:05:16.877191 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:16.877169 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-bundle" (OuterVolumeSpecName: "bundle") pod "47f8d6e2-adb4-4a49-8da5-03939d9c4e69" (UID: "47f8d6e2-adb4-4a49-8da5-03939d9c4e69"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:05:16.878868 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:16.878838 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-kube-api-access-66n4c" (OuterVolumeSpecName: "kube-api-access-66n4c") pod "47f8d6e2-adb4-4a49-8da5-03939d9c4e69" (UID: "47f8d6e2-adb4-4a49-8da5-03939d9c4e69"). InnerVolumeSpecName "kube-api-access-66n4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:05:16.883336 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:16.883308 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-util" (OuterVolumeSpecName: "util") pod "47f8d6e2-adb4-4a49-8da5-03939d9c4e69" (UID: "47f8d6e2-adb4-4a49-8da5-03939d9c4e69"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:05:16.978131 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:16.978027 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-util\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:05:16.978131 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:16.978072 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66n4c\" (UniqueName: \"kubernetes.io/projected/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-kube-api-access-66n4c\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:05:16.978131 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:16.978088 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47f8d6e2-adb4-4a49-8da5-03939d9c4e69-bundle\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:05:17.685526 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:17.685489 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" event={"ID":"47f8d6e2-adb4-4a49-8da5-03939d9c4e69","Type":"ContainerDied","Data":"24a41e9bbe808954530efcab4f2f4362339dfe2fa3cbd46ba1bff1005b98bcfa"} Jul 21 18:05:17.685526 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:17.685509 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f89684" Jul 21 18:05:17.685731 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:17.685536 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24a41e9bbe808954530efcab4f2f4362339dfe2fa3cbd46ba1bff1005b98bcfa" Jul 21 18:05:21.415217 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:21.415187 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:23.416569 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:23.416531 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:26.579970 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:26.579917 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:26.740191 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:26.740163 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:28.115488 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.115452 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv"] Jul 21 18:05:28.115839 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.115739 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47f8d6e2-adb4-4a49-8da5-03939d9c4e69" containerName="extract" Jul 21 18:05:28.115839 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.115750 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="47f8d6e2-adb4-4a49-8da5-03939d9c4e69" containerName="extract" Jul 21 18:05:28.115839 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.115769 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47f8d6e2-adb4-4a49-8da5-03939d9c4e69" containerName="util" Jul 21 18:05:28.115839 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.115774 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="47f8d6e2-adb4-4a49-8da5-03939d9c4e69" containerName="util" Jul 21 18:05:28.115839 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.115779 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47f8d6e2-adb4-4a49-8da5-03939d9c4e69" containerName="pull" Jul 21 18:05:28.115839 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.115785 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="47f8d6e2-adb4-4a49-8da5-03939d9c4e69" containerName="pull" Jul 21 18:05:28.115839 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.115837 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="47f8d6e2-adb4-4a49-8da5-03939d9c4e69" containerName="extract" Jul 21 18:05:28.118851 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.118834 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" Jul 21 18:05:28.121278 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.121261 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p47rs\"" Jul 21 18:05:28.121354 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.121266 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 21 18:05:28.122172 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.122154 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 21 18:05:28.127443 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.127424 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv"] Jul 21 18:05:28.173398 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.173366 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62fdn\" (UniqueName: \"kubernetes.io/projected/d001d931-9aaf-42aa-b268-33568aac68a0-kube-api-access-62fdn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv\" (UID: \"d001d931-9aaf-42aa-b268-33568aac68a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" Jul 21 18:05:28.173594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.173414 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d001d931-9aaf-42aa-b268-33568aac68a0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv\" (UID: \"d001d931-9aaf-42aa-b268-33568aac68a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" Jul 21 18:05:28.173594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.173477 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d001d931-9aaf-42aa-b268-33568aac68a0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv\" (UID: \"d001d931-9aaf-42aa-b268-33568aac68a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" Jul 21 18:05:28.274753 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.274711 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62fdn\" (UniqueName: \"kubernetes.io/projected/d001d931-9aaf-42aa-b268-33568aac68a0-kube-api-access-62fdn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv\" (UID: \"d001d931-9aaf-42aa-b268-33568aac68a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" Jul 21 18:05:28.274975 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.274762 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d001d931-9aaf-42aa-b268-33568aac68a0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv\" (UID: \"d001d931-9aaf-42aa-b268-33568aac68a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" Jul 21 18:05:28.274975 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.274790 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d001d931-9aaf-42aa-b268-33568aac68a0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv\" (UID: \"d001d931-9aaf-42aa-b268-33568aac68a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" Jul 21 18:05:28.275214 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.275192 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d001d931-9aaf-42aa-b268-33568aac68a0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv\" (UID: \"d001d931-9aaf-42aa-b268-33568aac68a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" Jul 21 18:05:28.275273 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.275247 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d001d931-9aaf-42aa-b268-33568aac68a0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv\" (UID: \"d001d931-9aaf-42aa-b268-33568aac68a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" Jul 21 18:05:28.283730 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.283701 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62fdn\" (UniqueName: \"kubernetes.io/projected/d001d931-9aaf-42aa-b268-33568aac68a0-kube-api-access-62fdn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv\" (UID: \"d001d931-9aaf-42aa-b268-33568aac68a0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" Jul 21 18:05:28.428725 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.428622 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" Jul 21 18:05:28.562043 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.562011 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv"] Jul 21 18:05:28.564915 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:05:28.564879 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd001d931_9aaf_42aa_b268_33568aac68a0.slice/crio-a86f308c1341122c79505015f06728edde7119f98752c5a0b73562f155319f57 WatchSource:0}: Error finding container a86f308c1341122c79505015f06728edde7119f98752c5a0b73562f155319f57: Status 404 returned error can't find the container with id a86f308c1341122c79505015f06728edde7119f98752c5a0b73562f155319f57 Jul 21 18:05:28.622767 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.622737 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:28.734106 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.734023 2568 generic.go:356] "Generic (PLEG): container finished" podID="d001d931-9aaf-42aa-b268-33568aac68a0" containerID="1b4afaa060d09eb8bc642d62e18cb4caedc1aa53fb404eda60074687e0aabaa8" exitCode=0 Jul 21 18:05:28.734252 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.734122 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" event={"ID":"d001d931-9aaf-42aa-b268-33568aac68a0","Type":"ContainerDied","Data":"1b4afaa060d09eb8bc642d62e18cb4caedc1aa53fb404eda60074687e0aabaa8"} Jul 21 18:05:28.734252 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.734182 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" event={"ID":"d001d931-9aaf-42aa-b268-33568aac68a0","Type":"ContainerStarted","Data":"a86f308c1341122c79505015f06728edde7119f98752c5a0b73562f155319f57"} Jul 21 18:05:28.734689 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:28.734675 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 18:05:29.740734 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:29.740699 2568 generic.go:356] "Generic (PLEG): container finished" podID="d001d931-9aaf-42aa-b268-33568aac68a0" containerID="7b65315f1d903fe0b9e092e1676d03752fc2740a599885920a3f25d99568de38" exitCode=0 Jul 21 18:05:29.741114 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:29.740761 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" event={"ID":"d001d931-9aaf-42aa-b268-33568aac68a0","Type":"ContainerDied","Data":"7b65315f1d903fe0b9e092e1676d03752fc2740a599885920a3f25d99568de38"} Jul 21 18:05:30.748684 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:30.748648 2568 generic.go:356] "Generic (PLEG): container finished" podID="d001d931-9aaf-42aa-b268-33568aac68a0" containerID="7a2200e62f8699efdd039903898cb2982ae8f9813cdc26aee5dc444c3020dc4c" exitCode=0 Jul 21 18:05:30.749105 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:30.748697 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" event={"ID":"d001d931-9aaf-42aa-b268-33568aac68a0","Type":"ContainerDied","Data":"7a2200e62f8699efdd039903898cb2982ae8f9813cdc26aee5dc444c3020dc4c"} Jul 21 18:05:31.890760 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:31.890735 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" Jul 21 18:05:32.007281 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:32.007198 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d001d931-9aaf-42aa-b268-33568aac68a0-bundle\") pod \"d001d931-9aaf-42aa-b268-33568aac68a0\" (UID: \"d001d931-9aaf-42aa-b268-33568aac68a0\") " Jul 21 18:05:32.007281 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:32.007242 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d001d931-9aaf-42aa-b268-33568aac68a0-util\") pod \"d001d931-9aaf-42aa-b268-33568aac68a0\" (UID: \"d001d931-9aaf-42aa-b268-33568aac68a0\") " Jul 21 18:05:32.007281 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:32.007274 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62fdn\" (UniqueName: \"kubernetes.io/projected/d001d931-9aaf-42aa-b268-33568aac68a0-kube-api-access-62fdn\") pod \"d001d931-9aaf-42aa-b268-33568aac68a0\" (UID: \"d001d931-9aaf-42aa-b268-33568aac68a0\") " Jul 21 18:05:32.008005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:32.007970 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d001d931-9aaf-42aa-b268-33568aac68a0-bundle" (OuterVolumeSpecName: "bundle") pod "d001d931-9aaf-42aa-b268-33568aac68a0" (UID: "d001d931-9aaf-42aa-b268-33568aac68a0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:05:32.009379 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:32.009355 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d001d931-9aaf-42aa-b268-33568aac68a0-kube-api-access-62fdn" (OuterVolumeSpecName: "kube-api-access-62fdn") pod "d001d931-9aaf-42aa-b268-33568aac68a0" (UID: "d001d931-9aaf-42aa-b268-33568aac68a0"). InnerVolumeSpecName "kube-api-access-62fdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:05:32.013097 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:32.013060 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d001d931-9aaf-42aa-b268-33568aac68a0-util" (OuterVolumeSpecName: "util") pod "d001d931-9aaf-42aa-b268-33568aac68a0" (UID: "d001d931-9aaf-42aa-b268-33568aac68a0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:05:32.108797 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:32.108760 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62fdn\" (UniqueName: \"kubernetes.io/projected/d001d931-9aaf-42aa-b268-33568aac68a0-kube-api-access-62fdn\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:05:32.108797 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:32.108791 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d001d931-9aaf-42aa-b268-33568aac68a0-bundle\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:05:32.108797 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:32.108800 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d001d931-9aaf-42aa-b268-33568aac68a0-util\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:05:32.759762 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:32.759733 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" Jul 21 18:05:32.759929 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:32.759777 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvswgv" event={"ID":"d001d931-9aaf-42aa-b268-33568aac68a0","Type":"ContainerDied","Data":"a86f308c1341122c79505015f06728edde7119f98752c5a0b73562f155319f57"} Jul 21 18:05:32.759929 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:32.759818 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a86f308c1341122c79505015f06728edde7119f98752c5a0b73562f155319f57" Jul 21 18:05:32.989929 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:32.989884 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:33.067994 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.067907 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-754c46d99f-947qv"] Jul 21 18:05:33.068220 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.068208 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d001d931-9aaf-42aa-b268-33568aac68a0" containerName="util" Jul 21 18:05:33.068260 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.068222 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d001d931-9aaf-42aa-b268-33568aac68a0" containerName="util" Jul 21 18:05:33.068260 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.068237 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d001d931-9aaf-42aa-b268-33568aac68a0" containerName="pull" Jul 21 18:05:33.068260 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.068242 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d001d931-9aaf-42aa-b268-33568aac68a0" containerName="pull" Jul 21 18:05:33.068260 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.068254 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d001d931-9aaf-42aa-b268-33568aac68a0" containerName="extract" Jul 21 18:05:33.068260 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.068259 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d001d931-9aaf-42aa-b268-33568aac68a0" containerName="extract" Jul 21 18:05:33.068398 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.068305 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d001d931-9aaf-42aa-b268-33568aac68a0" containerName="extract" Jul 21 18:05:33.071259 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.071239 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:33.075236 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.075209 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 21 18:05:33.075384 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.075365 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 21 18:05:33.075848 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.075829 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 21 18:05:33.078613 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.078596 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 21 18:05:33.078708 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.078613 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 21 18:05:33.079122 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.079103 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4m2hr\"" Jul 21 18:05:33.096781 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.096748 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-754c46d99f-947qv"] Jul 21 18:05:33.103711 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.103682 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:33.217816 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.217781 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/89d7a857-45f8-466d-97bc-19b9235ca4fc-metrics-cert\") pod \"lws-controller-manager-754c46d99f-947qv\" (UID: \"89d7a857-45f8-466d-97bc-19b9235ca4fc\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:33.218000 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.217841 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jljtf\" (UniqueName: \"kubernetes.io/projected/89d7a857-45f8-466d-97bc-19b9235ca4fc-kube-api-access-jljtf\") pod \"lws-controller-manager-754c46d99f-947qv\" (UID: \"89d7a857-45f8-466d-97bc-19b9235ca4fc\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:33.218000 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.217867 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/89d7a857-45f8-466d-97bc-19b9235ca4fc-manager-config\") pod \"lws-controller-manager-754c46d99f-947qv\" (UID: \"89d7a857-45f8-466d-97bc-19b9235ca4fc\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:33.218000 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.217890 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/89d7a857-45f8-466d-97bc-19b9235ca4fc-cert\") pod \"lws-controller-manager-754c46d99f-947qv\" (UID: \"89d7a857-45f8-466d-97bc-19b9235ca4fc\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:33.318648 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.318551 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/89d7a857-45f8-466d-97bc-19b9235ca4fc-metrics-cert\") pod \"lws-controller-manager-754c46d99f-947qv\" (UID: \"89d7a857-45f8-466d-97bc-19b9235ca4fc\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:33.318648 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.318622 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jljtf\" (UniqueName: \"kubernetes.io/projected/89d7a857-45f8-466d-97bc-19b9235ca4fc-kube-api-access-jljtf\") pod \"lws-controller-manager-754c46d99f-947qv\" (UID: \"89d7a857-45f8-466d-97bc-19b9235ca4fc\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:33.318873 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.318665 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/89d7a857-45f8-466d-97bc-19b9235ca4fc-manager-config\") pod \"lws-controller-manager-754c46d99f-947qv\" (UID: \"89d7a857-45f8-466d-97bc-19b9235ca4fc\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:33.318873 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.318690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/89d7a857-45f8-466d-97bc-19b9235ca4fc-cert\") pod \"lws-controller-manager-754c46d99f-947qv\" (UID: \"89d7a857-45f8-466d-97bc-19b9235ca4fc\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:33.319382 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.319360 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/89d7a857-45f8-466d-97bc-19b9235ca4fc-manager-config\") pod \"lws-controller-manager-754c46d99f-947qv\" (UID: \"89d7a857-45f8-466d-97bc-19b9235ca4fc\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:33.321237 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.321213 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/89d7a857-45f8-466d-97bc-19b9235ca4fc-cert\") pod \"lws-controller-manager-754c46d99f-947qv\" (UID: \"89d7a857-45f8-466d-97bc-19b9235ca4fc\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:33.321343 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.321221 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/89d7a857-45f8-466d-97bc-19b9235ca4fc-metrics-cert\") pod \"lws-controller-manager-754c46d99f-947qv\" (UID: \"89d7a857-45f8-466d-97bc-19b9235ca4fc\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:33.327263 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.327241 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jljtf\" (UniqueName: \"kubernetes.io/projected/89d7a857-45f8-466d-97bc-19b9235ca4fc-kube-api-access-jljtf\") pod \"lws-controller-manager-754c46d99f-947qv\" (UID: \"89d7a857-45f8-466d-97bc-19b9235ca4fc\") " pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:33.380957 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.380888 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:33.514252 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.514226 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-754c46d99f-947qv"] Jul 21 18:05:33.516555 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:05:33.516528 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89d7a857_45f8_466d_97bc_19b9235ca4fc.slice/crio-a93824ecdc40c1512ddbac5b494ecc5f42aba8a52bdbfcadb3de609a1bcb7ed3 WatchSource:0}: Error finding container a93824ecdc40c1512ddbac5b494ecc5f42aba8a52bdbfcadb3de609a1bcb7ed3: Status 404 returned error can't find the container with id a93824ecdc40c1512ddbac5b494ecc5f42aba8a52bdbfcadb3de609a1bcb7ed3 Jul 21 18:05:33.765107 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:33.765067 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" event={"ID":"89d7a857-45f8-466d-97bc-19b9235ca4fc","Type":"ContainerStarted","Data":"a93824ecdc40c1512ddbac5b494ecc5f42aba8a52bdbfcadb3de609a1bcb7ed3"} Jul 21 18:05:35.775331 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:35.775295 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" event={"ID":"89d7a857-45f8-466d-97bc-19b9235ca4fc","Type":"ContainerStarted","Data":"5a50e87ef8161cc1103c913eb736ab6ad984ef435ec6c3cceb233301826cbc65"} Jul 21 18:05:35.775984 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:35.775411 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:41.024200 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:41.024161 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:41.121333 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:41.121295 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:41.694265 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:41.694238 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:42.053621 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.053534 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:42.156731 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.156698 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:42.246758 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.246702 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" podStartSLOduration=7.871650174 podStartE2EDuration="9.246684516s" totalImagesPullingTime="1.375034342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:05:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:05:35.805399405 +0000 UTC m=+337.073710846" watchObservedRunningTime="2026-07-21 18:05:42.246684516 +0000 UTC m=+343.514995956" Jul 21 18:05:42.248135 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.248114 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9"] Jul 21 18:05:42.251641 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.251624 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.254746 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.254727 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 21 18:05:42.255158 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.255138 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 21 18:05:42.255251 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.255236 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7tzw6\"" Jul 21 18:05:42.255374 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.255355 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 21 18:05:42.258879 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.258860 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 21 18:05:42.305047 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.304967 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9"] Jul 21 18:05:42.397466 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.397428 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.397622 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.397481 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-manifests\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.397622 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.397502 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5v5x\" (UniqueName: \"kubernetes.io/projected/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-kube-api-access-q5v5x\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.397622 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.397559 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-tmp\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.397622 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.397587 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.434440 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.434399 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:42.498833 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.498800 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-manifests\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.498833 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.498832 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5v5x\" (UniqueName: \"kubernetes.io/projected/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-kube-api-access-q5v5x\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.499090 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.498860 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-tmp\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.499090 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.498883 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.499090 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.498992 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.499240 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.499199 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-manifests\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.501123 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.501104 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-tmp\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.501405 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.501388 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.501445 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.501419 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.513338 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.513313 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5v5x\" (UniqueName: \"kubernetes.io/projected/1cdfb9a3-3beb-4685-817e-f17651d1fb1c-kube-api-access-q5v5x\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-n92j9\" (UID: \"1cdfb9a3-3beb-4685-817e-f17651d1fb1c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.550798 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.550765 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:42.561717 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.561649 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:42.660387 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.660350 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:42.716102 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.716072 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9"] Jul 21 18:05:42.718326 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:05:42.718304 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cdfb9a3_3beb_4685_817e_f17651d1fb1c.slice/crio-961c9d3fe5cf9bdb329d45b2a9943b1d56262bf432deb07b37f8617c0a07bf69 WatchSource:0}: Error finding container 961c9d3fe5cf9bdb329d45b2a9943b1d56262bf432deb07b37f8617c0a07bf69: Status 404 returned error can't find the container with id 961c9d3fe5cf9bdb329d45b2a9943b1d56262bf432deb07b37f8617c0a07bf69 Jul 21 18:05:42.769564 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.769525 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:42.807767 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.807737 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" event={"ID":"1cdfb9a3-3beb-4685-817e-f17651d1fb1c","Type":"ContainerStarted","Data":"961c9d3fe5cf9bdb329d45b2a9943b1d56262bf432deb07b37f8617c0a07bf69"} Jul 21 18:05:42.918742 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:42.918707 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:05:45.824664 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:45.824622 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" event={"ID":"1cdfb9a3-3beb-4685-817e-f17651d1fb1c","Type":"ContainerStarted","Data":"19d0078fe41378702112e522f7ca7a9109a4751c6b97b9a9788864b13b25d3a3"} Jul 21 18:05:46.782476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:46.782442 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-754c46d99f-947qv" Jul 21 18:05:46.831056 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:46.831026 2568 generic.go:356] "Generic (PLEG): container finished" podID="1cdfb9a3-3beb-4685-817e-f17651d1fb1c" containerID="19d0078fe41378702112e522f7ca7a9109a4751c6b97b9a9788864b13b25d3a3" exitCode=0 Jul 21 18:05:46.831418 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:46.831120 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" event={"ID":"1cdfb9a3-3beb-4685-817e-f17651d1fb1c","Type":"ContainerDied","Data":"19d0078fe41378702112e522f7ca7a9109a4751c6b97b9a9788864b13b25d3a3"} Jul 21 18:05:47.837749 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:47.837714 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" event={"ID":"1cdfb9a3-3beb-4685-817e-f17651d1fb1c","Type":"ContainerStarted","Data":"73f338bf4d18c03955c6dbd70e7907cbdb4633b13897d62e4335b566e36d41be"} Jul 21 18:05:47.838163 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:47.837804 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:05:47.865962 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:47.865883 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" podStartSLOduration=1.539604136 podStartE2EDuration="5.865863231s" totalImagesPullingTime="3.264399574s" totalInitContainerRuntime="1.061859521s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:05:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:05:47.865821585 +0000 UTC m=+349.134133027" watchObservedRunningTime="2026-07-21 18:05:47.865863231 +0000 UTC m=+349.134174670" Jul 21 18:05:58.844879 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:05:58.844790 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-n92j9" Jul 21 18:06:01.014215 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:01.014170 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:01.142653 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:01.142616 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:01.632616 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:01.632584 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:01.919217 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:01.919132 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:02.137546 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:02.137509 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:02.294869 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:02.294775 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:02.459176 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:02.459117 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:03.136199 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:03.136166 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:03.345767 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:03.345726 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:03.527329 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:03.527239 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:03.790236 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:03.790149 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:03.893884 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:03.893850 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:05.460985 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:05.460930 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:09.817620 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.817583 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v"] Jul 21 18:06:09.824118 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.824096 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:09.826733 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.826710 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 21 18:06:09.826866 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.826710 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 21 18:06:09.827009 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.826991 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 21 18:06:09.827080 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.827058 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 21 18:06:09.827191 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.827175 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-76dl7\"" Jul 21 18:06:09.841915 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.841887 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v"] Jul 21 18:06:09.919931 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.919892 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cd4980f8-00b8-4449-96f6-f409c3d52c89-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:09.920143 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.919963 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4980f8-00b8-4449-96f6-f409c3d52c89-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:09.920143 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.919990 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cd4980f8-00b8-4449-96f6-f409c3d52c89-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:09.920143 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.920013 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cd4980f8-00b8-4449-96f6-f409c3d52c89-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:09.920143 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.920029 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4980f8-00b8-4449-96f6-f409c3d52c89-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:09.920143 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.920114 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cd4980f8-00b8-4449-96f6-f409c3d52c89-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:09.920336 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:09.920152 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twgjq\" (UniqueName: \"kubernetes.io/projected/cd4980f8-00b8-4449-96f6-f409c3d52c89-kube-api-access-twgjq\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.021571 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.021524 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cd4980f8-00b8-4449-96f6-f409c3d52c89-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.021778 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.021582 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cd4980f8-00b8-4449-96f6-f409c3d52c89-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.021778 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.021611 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4980f8-00b8-4449-96f6-f409c3d52c89-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.021778 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.021664 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cd4980f8-00b8-4449-96f6-f409c3d52c89-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.021778 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.021687 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twgjq\" (UniqueName: \"kubernetes.io/projected/cd4980f8-00b8-4449-96f6-f409c3d52c89-kube-api-access-twgjq\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.021778 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.021731 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cd4980f8-00b8-4449-96f6-f409c3d52c89-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.022081 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.021783 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4980f8-00b8-4449-96f6-f409c3d52c89-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.022431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.022401 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cd4980f8-00b8-4449-96f6-f409c3d52c89-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.023926 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.023904 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4980f8-00b8-4449-96f6-f409c3d52c89-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.024114 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.024094 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cd4980f8-00b8-4449-96f6-f409c3d52c89-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.024358 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.024338 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cd4980f8-00b8-4449-96f6-f409c3d52c89-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.024463 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.024440 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cd4980f8-00b8-4449-96f6-f409c3d52c89-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.034206 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.034182 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4980f8-00b8-4449-96f6-f409c3d52c89-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.035130 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.035112 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twgjq\" (UniqueName: \"kubernetes.io/projected/cd4980f8-00b8-4449-96f6-f409c3d52c89-kube-api-access-twgjq\") pod \"istiod-openshift-gateway-94bb8fbfd-fbm4v\" (UID: \"cd4980f8-00b8-4449-96f6-f409c3d52c89\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.133651 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.133620 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:10.189406 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.189371 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:10.294887 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.294849 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v"] Jul 21 18:06:10.298879 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:06:10.298790 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd4980f8_00b8_4449_96f6_f409c3d52c89.slice/crio-a1722db36ef0e610715e2bce3296a1df67dbeeb7b1c2a5422a714707f8f3a1bf WatchSource:0}: Error finding container a1722db36ef0e610715e2bce3296a1df67dbeeb7b1c2a5422a714707f8f3a1bf: Status 404 returned error can't find the container with id a1722db36ef0e610715e2bce3296a1df67dbeeb7b1c2a5422a714707f8f3a1bf Jul 21 18:06:10.299226 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.299203 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:10.666830 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.666797 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:10.930265 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.930110 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:10.956867 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:10.956796 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" event={"ID":"cd4980f8-00b8-4449-96f6-f409c3d52c89","Type":"ContainerStarted","Data":"a1722db36ef0e610715e2bce3296a1df67dbeeb7b1c2a5422a714707f8f3a1bf"} Jul 21 18:06:11.126801 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:11.126718 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:12.533716 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:12.533671 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 21 18:06:12.534035 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:12.533758 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 21 18:06:12.968738 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:12.968706 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" event={"ID":"cd4980f8-00b8-4449-96f6-f409c3d52c89","Type":"ContainerStarted","Data":"0890383f16cf108ca088991b976e3f116774714328b82feac8dbcb0295921dc5"} Jul 21 18:06:12.968970 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:12.968776 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:13.013100 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:13.013043 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" podStartSLOduration=1.7807411960000001 podStartE2EDuration="4.013026548s" totalImagesPullingTime="2.232285352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:06:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:06:13.011808901 +0000 UTC m=+374.280120342" watchObservedRunningTime="2026-07-21 18:06:13.013026548 +0000 UTC m=+374.281337988" Jul 21 18:06:13.471722 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:13.471688 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:13.741536 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:13.741453 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:13.888922 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:13.888890 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:13.974705 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:13.974673 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fbm4v" Jul 21 18:06:13.994766 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:13.994680 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:14.146254 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:14.146207 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:41.987250 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:41.987214 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:42.078699 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:42.078654 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:43.691594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:43.691558 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:06:43.773616 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:06:43.773586 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:02.566843 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:02.566800 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:02.678308 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:02.678269 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:02.877773 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:02.877736 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:18.009378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.009335 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn"] Jul 21 18:07:18.013028 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.013004 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" Jul 21 18:07:18.017960 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.017912 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xcwkl\"" Jul 21 18:07:18.018111 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.017912 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 18:07:18.018111 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.017912 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 18:07:18.032224 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.032191 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn"] Jul 21 18:07:18.114525 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.114487 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94cf9b0f-b8bc-466b-91af-2cb153917fb4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn\" (UID: \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" Jul 21 18:07:18.114714 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.114549 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdl7j\" (UniqueName: \"kubernetes.io/projected/94cf9b0f-b8bc-466b-91af-2cb153917fb4-kube-api-access-gdl7j\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn\" (UID: \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" Jul 21 18:07:18.114714 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.114597 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94cf9b0f-b8bc-466b-91af-2cb153917fb4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn\" (UID: \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" Jul 21 18:07:18.215726 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.215687 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94cf9b0f-b8bc-466b-91af-2cb153917fb4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn\" (UID: \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" Jul 21 18:07:18.215896 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.215754 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94cf9b0f-b8bc-466b-91af-2cb153917fb4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn\" (UID: \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" Jul 21 18:07:18.215896 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.215790 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdl7j\" (UniqueName: \"kubernetes.io/projected/94cf9b0f-b8bc-466b-91af-2cb153917fb4-kube-api-access-gdl7j\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn\" (UID: \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" Jul 21 18:07:18.216125 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.216093 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94cf9b0f-b8bc-466b-91af-2cb153917fb4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn\" (UID: \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" Jul 21 18:07:18.216164 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.216142 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94cf9b0f-b8bc-466b-91af-2cb153917fb4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn\" (UID: \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" Jul 21 18:07:18.225311 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.225284 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdl7j\" (UniqueName: \"kubernetes.io/projected/94cf9b0f-b8bc-466b-91af-2cb153917fb4-kube-api-access-gdl7j\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn\" (UID: \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" Jul 21 18:07:18.322263 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.322173 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" Jul 21 18:07:18.463782 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.463754 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn"] Jul 21 18:07:18.466637 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:07:18.466610 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94cf9b0f_b8bc_466b_91af_2cb153917fb4.slice/crio-af0b76eac3e4b52e1b613374ae2b6e94fe0246b9cfb6520ee155ad4ac1ca5caf WatchSource:0}: Error finding container af0b76eac3e4b52e1b613374ae2b6e94fe0246b9cfb6520ee155ad4ac1ca5caf: Status 404 returned error can't find the container with id af0b76eac3e4b52e1b613374ae2b6e94fe0246b9cfb6520ee155ad4ac1ca5caf Jul 21 18:07:18.808151 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.808111 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt"] Jul 21 18:07:18.811514 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.811496 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" Jul 21 18:07:18.823965 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.823923 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt"] Jul 21 18:07:18.922197 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.922150 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt\" (UID: \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" Jul 21 18:07:18.922396 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.922203 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxj4t\" (UniqueName: \"kubernetes.io/projected/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-kube-api-access-vxj4t\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt\" (UID: \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" Jul 21 18:07:18.922396 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:18.922233 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt\" (UID: \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" Jul 21 18:07:19.023299 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.023256 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt\" (UID: \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" Jul 21 18:07:19.023299 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.023299 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxj4t\" (UniqueName: \"kubernetes.io/projected/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-kube-api-access-vxj4t\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt\" (UID: \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" Jul 21 18:07:19.023823 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.023321 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt\" (UID: \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" Jul 21 18:07:19.023823 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.023745 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt\" (UID: \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" Jul 21 18:07:19.023823 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.023750 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt\" (UID: \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" Jul 21 18:07:19.032153 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.032129 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxj4t\" (UniqueName: \"kubernetes.io/projected/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-kube-api-access-vxj4t\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt\" (UID: \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" Jul 21 18:07:19.120709 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.120670 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" Jul 21 18:07:19.263705 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.263682 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt"] Jul 21 18:07:19.266238 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:07:19.266207 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f68f4f1_d977_4296_a62e_e9fda2a9fd59.slice/crio-3bbcf4ac0ac1282063a218b55159e824c7b240a912af7fd1a78145784bcc2eb6 WatchSource:0}: Error finding container 3bbcf4ac0ac1282063a218b55159e824c7b240a912af7fd1a78145784bcc2eb6: Status 404 returned error can't find the container with id 3bbcf4ac0ac1282063a218b55159e824c7b240a912af7fd1a78145784bcc2eb6 Jul 21 18:07:19.279155 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.279126 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" event={"ID":"2f68f4f1-d977-4296-a62e-e9fda2a9fd59","Type":"ContainerStarted","Data":"3bbcf4ac0ac1282063a218b55159e824c7b240a912af7fd1a78145784bcc2eb6"} Jul 21 18:07:19.280668 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.280640 2568 generic.go:356] "Generic (PLEG): container finished" podID="94cf9b0f-b8bc-466b-91af-2cb153917fb4" containerID="902a57e919082d708734c1a1484dfd3eea7c25ec527a0de03ae51bec91384e59" exitCode=0 Jul 21 18:07:19.280780 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.280731 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" event={"ID":"94cf9b0f-b8bc-466b-91af-2cb153917fb4","Type":"ContainerDied","Data":"902a57e919082d708734c1a1484dfd3eea7c25ec527a0de03ae51bec91384e59"} Jul 21 18:07:19.280879 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.280778 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" event={"ID":"94cf9b0f-b8bc-466b-91af-2cb153917fb4","Type":"ContainerStarted","Data":"af0b76eac3e4b52e1b613374ae2b6e94fe0246b9cfb6520ee155ad4ac1ca5caf"} Jul 21 18:07:19.624666 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.624587 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx"] Jul 21 18:07:19.628117 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.628099 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" Jul 21 18:07:19.643153 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.643122 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx"] Jul 21 18:07:19.729017 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.728975 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82rq9\" (UniqueName: \"kubernetes.io/projected/0d401d4c-07a8-42c7-afeb-44ee744e346e-kube-api-access-82rq9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx\" (UID: \"0d401d4c-07a8-42c7-afeb-44ee744e346e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" Jul 21 18:07:19.729190 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.729028 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d401d4c-07a8-42c7-afeb-44ee744e346e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx\" (UID: \"0d401d4c-07a8-42c7-afeb-44ee744e346e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" Jul 21 18:07:19.729190 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.729113 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d401d4c-07a8-42c7-afeb-44ee744e346e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx\" (UID: \"0d401d4c-07a8-42c7-afeb-44ee744e346e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" Jul 21 18:07:19.830311 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.830267 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82rq9\" (UniqueName: \"kubernetes.io/projected/0d401d4c-07a8-42c7-afeb-44ee744e346e-kube-api-access-82rq9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx\" (UID: \"0d401d4c-07a8-42c7-afeb-44ee744e346e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" Jul 21 18:07:19.830494 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.830322 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d401d4c-07a8-42c7-afeb-44ee744e346e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx\" (UID: \"0d401d4c-07a8-42c7-afeb-44ee744e346e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" Jul 21 18:07:19.830494 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.830369 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d401d4c-07a8-42c7-afeb-44ee744e346e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx\" (UID: \"0d401d4c-07a8-42c7-afeb-44ee744e346e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" Jul 21 18:07:19.830817 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.830796 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d401d4c-07a8-42c7-afeb-44ee744e346e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx\" (UID: \"0d401d4c-07a8-42c7-afeb-44ee744e346e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" Jul 21 18:07:19.830855 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.830811 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d401d4c-07a8-42c7-afeb-44ee744e346e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx\" (UID: \"0d401d4c-07a8-42c7-afeb-44ee744e346e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" Jul 21 18:07:19.839289 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.839262 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82rq9\" (UniqueName: \"kubernetes.io/projected/0d401d4c-07a8-42c7-afeb-44ee744e346e-kube-api-access-82rq9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx\" (UID: \"0d401d4c-07a8-42c7-afeb-44ee744e346e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" Jul 21 18:07:19.937039 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:19.936919 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" Jul 21 18:07:20.034441 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.034404 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz"] Jul 21 18:07:20.038535 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.038514 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" Jul 21 18:07:20.052250 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.052220 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz"] Jul 21 18:07:20.092359 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.092334 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx"] Jul 21 18:07:20.094985 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:07:20.094921 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d401d4c_07a8_42c7_afeb_44ee744e346e.slice/crio-b62a8106fecb690d0bac8f812c0743228faac63f768a46f24bceef3810f0a93c WatchSource:0}: Error finding container b62a8106fecb690d0bac8f812c0743228faac63f768a46f24bceef3810f0a93c: Status 404 returned error can't find the container with id b62a8106fecb690d0bac8f812c0743228faac63f768a46f24bceef3810f0a93c Jul 21 18:07:20.132564 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.132533 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz\" (UID: \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" Jul 21 18:07:20.132564 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.132569 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26859\" (UniqueName: \"kubernetes.io/projected/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-kube-api-access-26859\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz\" (UID: \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" Jul 21 18:07:20.132752 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.132613 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz\" (UID: \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" Jul 21 18:07:20.233701 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.233670 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz\" (UID: \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" Jul 21 18:07:20.233824 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.233721 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26859\" (UniqueName: \"kubernetes.io/projected/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-kube-api-access-26859\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz\" (UID: \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" Jul 21 18:07:20.233824 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.233800 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz\" (UID: \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" Jul 21 18:07:20.234102 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.234079 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz\" (UID: \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" Jul 21 18:07:20.234241 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.234172 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz\" (UID: \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" Jul 21 18:07:20.244761 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.244730 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26859\" (UniqueName: \"kubernetes.io/projected/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-kube-api-access-26859\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz\" (UID: \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" Jul 21 18:07:20.287904 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.287869 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" event={"ID":"0d401d4c-07a8-42c7-afeb-44ee744e346e","Type":"ContainerStarted","Data":"b62a8106fecb690d0bac8f812c0743228faac63f768a46f24bceef3810f0a93c"} Jul 21 18:07:20.289531 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.289507 2568 generic.go:356] "Generic (PLEG): container finished" podID="2f68f4f1-d977-4296-a62e-e9fda2a9fd59" containerID="41cfd6650f20be38d3b99fe49a7aff78b3b167d4c76b1ce2bceee6d929b414ee" exitCode=0 Jul 21 18:07:20.289647 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.289559 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" event={"ID":"2f68f4f1-d977-4296-a62e-e9fda2a9fd59","Type":"ContainerDied","Data":"41cfd6650f20be38d3b99fe49a7aff78b3b167d4c76b1ce2bceee6d929b414ee"} Jul 21 18:07:20.349850 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.349819 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" Jul 21 18:07:20.516677 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:20.516653 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz"] Jul 21 18:07:20.518399 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:07:20.518371 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1023893_fbc7_47d0_a023_a9f0a76b6ccd.slice/crio-f2ec103340ca761e3b70d8ae689693e5fabe7573bd2868a43c1b68f77b73ab9d WatchSource:0}: Error finding container f2ec103340ca761e3b70d8ae689693e5fabe7573bd2868a43c1b68f77b73ab9d: Status 404 returned error can't find the container with id f2ec103340ca761e3b70d8ae689693e5fabe7573bd2868a43c1b68f77b73ab9d Jul 21 18:07:21.296096 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:21.296056 2568 generic.go:356] "Generic (PLEG): container finished" podID="2f68f4f1-d977-4296-a62e-e9fda2a9fd59" containerID="9c1b257522c23fd429ddfb92cb6221433febdf662c5dc9a430ba1495cba91182" exitCode=0 Jul 21 18:07:21.296477 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:21.296140 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" event={"ID":"2f68f4f1-d977-4296-a62e-e9fda2a9fd59","Type":"ContainerDied","Data":"9c1b257522c23fd429ddfb92cb6221433febdf662c5dc9a430ba1495cba91182"} Jul 21 18:07:21.298095 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:21.298070 2568 generic.go:356] "Generic (PLEG): container finished" podID="94cf9b0f-b8bc-466b-91af-2cb153917fb4" containerID="cec7785b30ec7f7d41cd089a76f34549a2aab5e3408f0e24b41a8f32c8876f00" exitCode=0 Jul 21 18:07:21.298185 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:21.298150 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" event={"ID":"94cf9b0f-b8bc-466b-91af-2cb153917fb4","Type":"ContainerDied","Data":"cec7785b30ec7f7d41cd089a76f34549a2aab5e3408f0e24b41a8f32c8876f00"} Jul 21 18:07:21.299720 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:21.299699 2568 generic.go:356] "Generic (PLEG): container finished" podID="0d401d4c-07a8-42c7-afeb-44ee744e346e" containerID="8d420b340a4811d9f8642b4e56b23525a72a50482d71dd8cbe2a0db3093510cc" exitCode=0 Jul 21 18:07:21.299788 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:21.299757 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" event={"ID":"0d401d4c-07a8-42c7-afeb-44ee744e346e","Type":"ContainerDied","Data":"8d420b340a4811d9f8642b4e56b23525a72a50482d71dd8cbe2a0db3093510cc"} Jul 21 18:07:21.301446 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:21.301424 2568 generic.go:356] "Generic (PLEG): container finished" podID="b1023893-fbc7-47d0-a023-a9f0a76b6ccd" containerID="abbd94090e1cb559fccaffdbbc48320779fa186c2f489a8432df5e71e16b9f96" exitCode=0 Jul 21 18:07:21.301538 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:21.301514 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" event={"ID":"b1023893-fbc7-47d0-a023-a9f0a76b6ccd","Type":"ContainerDied","Data":"abbd94090e1cb559fccaffdbbc48320779fa186c2f489a8432df5e71e16b9f96"} Jul 21 18:07:21.301594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:21.301567 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" event={"ID":"b1023893-fbc7-47d0-a023-a9f0a76b6ccd","Type":"ContainerStarted","Data":"f2ec103340ca761e3b70d8ae689693e5fabe7573bd2868a43c1b68f77b73ab9d"} Jul 21 18:07:22.309071 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:22.309032 2568 generic.go:356] "Generic (PLEG): container finished" podID="2f68f4f1-d977-4296-a62e-e9fda2a9fd59" containerID="2b2dae40995f96aa14639e6345f642b1a3988f576803b72b45ad3027be8ceb84" exitCode=0 Jul 21 18:07:22.309499 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:22.309122 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" event={"ID":"2f68f4f1-d977-4296-a62e-e9fda2a9fd59","Type":"ContainerDied","Data":"2b2dae40995f96aa14639e6345f642b1a3988f576803b72b45ad3027be8ceb84"} Jul 21 18:07:22.311147 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:22.311121 2568 generic.go:356] "Generic (PLEG): container finished" podID="94cf9b0f-b8bc-466b-91af-2cb153917fb4" containerID="2e2490e7585c64396f02ab5824062f98cd71923dc5b89aca2f061770a4a267bb" exitCode=0 Jul 21 18:07:22.311234 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:22.311197 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" event={"ID":"94cf9b0f-b8bc-466b-91af-2cb153917fb4","Type":"ContainerDied","Data":"2e2490e7585c64396f02ab5824062f98cd71923dc5b89aca2f061770a4a267bb"} Jul 21 18:07:22.312974 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:22.312951 2568 generic.go:356] "Generic (PLEG): container finished" podID="0d401d4c-07a8-42c7-afeb-44ee744e346e" containerID="610ca6c9da5fdf4af9493842f5a23c942b3fc2108d20f668a5eae791c6cb4984" exitCode=0 Jul 21 18:07:22.313085 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:22.313030 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" event={"ID":"0d401d4c-07a8-42c7-afeb-44ee744e346e","Type":"ContainerDied","Data":"610ca6c9da5fdf4af9493842f5a23c942b3fc2108d20f668a5eae791c6cb4984"} Jul 21 18:07:22.314816 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:22.314796 2568 generic.go:356] "Generic (PLEG): container finished" podID="b1023893-fbc7-47d0-a023-a9f0a76b6ccd" containerID="a1d19fea9a8a31df3d72020f63ce7b7391ce0f50564f855f00dc11817f003f77" exitCode=0 Jul 21 18:07:22.314884 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:22.314831 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" event={"ID":"b1023893-fbc7-47d0-a023-a9f0a76b6ccd","Type":"ContainerDied","Data":"a1d19fea9a8a31df3d72020f63ce7b7391ce0f50564f855f00dc11817f003f77"} Jul 21 18:07:23.322114 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.322072 2568 generic.go:356] "Generic (PLEG): container finished" podID="0d401d4c-07a8-42c7-afeb-44ee744e346e" containerID="4b58b5e419f441ca914145e54f681e1b724bcc96602235f48ea339eda9855d87" exitCode=0 Jul 21 18:07:23.322524 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.322155 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" event={"ID":"0d401d4c-07a8-42c7-afeb-44ee744e346e","Type":"ContainerDied","Data":"4b58b5e419f441ca914145e54f681e1b724bcc96602235f48ea339eda9855d87"} Jul 21 18:07:23.324102 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.324081 2568 generic.go:356] "Generic (PLEG): container finished" podID="b1023893-fbc7-47d0-a023-a9f0a76b6ccd" containerID="3130f316e1c9d372732f05f3e0e30eb5796f05357241d25aba2282746e271e48" exitCode=0 Jul 21 18:07:23.324178 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.324160 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" event={"ID":"b1023893-fbc7-47d0-a023-a9f0a76b6ccd","Type":"ContainerDied","Data":"3130f316e1c9d372732f05f3e0e30eb5796f05357241d25aba2282746e271e48"} Jul 21 18:07:23.468409 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.468376 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" Jul 21 18:07:23.502770 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.502741 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" Jul 21 18:07:23.565026 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.564992 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-util\") pod \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\" (UID: \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\") " Jul 21 18:07:23.565026 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.565030 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94cf9b0f-b8bc-466b-91af-2cb153917fb4-util\") pod \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\" (UID: \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\") " Jul 21 18:07:23.565237 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.565073 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-bundle\") pod \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\" (UID: \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\") " Jul 21 18:07:23.565237 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.565106 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxj4t\" (UniqueName: \"kubernetes.io/projected/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-kube-api-access-vxj4t\") pod \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\" (UID: \"2f68f4f1-d977-4296-a62e-e9fda2a9fd59\") " Jul 21 18:07:23.565237 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.565138 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94cf9b0f-b8bc-466b-91af-2cb153917fb4-bundle\") pod \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\" (UID: \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\") " Jul 21 18:07:23.565237 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.565167 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdl7j\" (UniqueName: \"kubernetes.io/projected/94cf9b0f-b8bc-466b-91af-2cb153917fb4-kube-api-access-gdl7j\") pod \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\" (UID: \"94cf9b0f-b8bc-466b-91af-2cb153917fb4\") " Jul 21 18:07:23.565704 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.565667 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-bundle" (OuterVolumeSpecName: "bundle") pod "2f68f4f1-d977-4296-a62e-e9fda2a9fd59" (UID: "2f68f4f1-d977-4296-a62e-e9fda2a9fd59"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:07:23.565704 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.565688 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94cf9b0f-b8bc-466b-91af-2cb153917fb4-bundle" (OuterVolumeSpecName: "bundle") pod "94cf9b0f-b8bc-466b-91af-2cb153917fb4" (UID: "94cf9b0f-b8bc-466b-91af-2cb153917fb4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:07:23.567454 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.567424 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94cf9b0f-b8bc-466b-91af-2cb153917fb4-kube-api-access-gdl7j" (OuterVolumeSpecName: "kube-api-access-gdl7j") pod "94cf9b0f-b8bc-466b-91af-2cb153917fb4" (UID: "94cf9b0f-b8bc-466b-91af-2cb153917fb4"). InnerVolumeSpecName "kube-api-access-gdl7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:07:23.567565 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.567522 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-kube-api-access-vxj4t" (OuterVolumeSpecName: "kube-api-access-vxj4t") pod "2f68f4f1-d977-4296-a62e-e9fda2a9fd59" (UID: "2f68f4f1-d977-4296-a62e-e9fda2a9fd59"). InnerVolumeSpecName "kube-api-access-vxj4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:07:23.570273 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.570239 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94cf9b0f-b8bc-466b-91af-2cb153917fb4-util" (OuterVolumeSpecName: "util") pod "94cf9b0f-b8bc-466b-91af-2cb153917fb4" (UID: "94cf9b0f-b8bc-466b-91af-2cb153917fb4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:07:23.570747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.570728 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-util" (OuterVolumeSpecName: "util") pod "2f68f4f1-d977-4296-a62e-e9fda2a9fd59" (UID: "2f68f4f1-d977-4296-a62e-e9fda2a9fd59"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:07:23.666503 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.666464 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-bundle\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:07:23.666503 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.666496 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxj4t\" (UniqueName: \"kubernetes.io/projected/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-kube-api-access-vxj4t\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:07:23.666503 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.666510 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94cf9b0f-b8bc-466b-91af-2cb153917fb4-bundle\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:07:23.666734 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.666521 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdl7j\" (UniqueName: \"kubernetes.io/projected/94cf9b0f-b8bc-466b-91af-2cb153917fb4-kube-api-access-gdl7j\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:07:23.666734 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.666533 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f68f4f1-d977-4296-a62e-e9fda2a9fd59-util\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:07:23.666734 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:23.666544 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94cf9b0f-b8bc-466b-91af-2cb153917fb4-util\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:07:24.331954 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.331908 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" event={"ID":"2f68f4f1-d977-4296-a62e-e9fda2a9fd59","Type":"ContainerDied","Data":"3bbcf4ac0ac1282063a218b55159e824c7b240a912af7fd1a78145784bcc2eb6"} Jul 21 18:07:24.332391 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.331981 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3bbcf4ac0ac1282063a218b55159e824c7b240a912af7fd1a78145784bcc2eb6" Jul 21 18:07:24.332391 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.331920 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt" Jul 21 18:07:24.333829 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.333798 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" event={"ID":"94cf9b0f-b8bc-466b-91af-2cb153917fb4","Type":"ContainerDied","Data":"af0b76eac3e4b52e1b613374ae2b6e94fe0246b9cfb6520ee155ad4ac1ca5caf"} Jul 21 18:07:24.333829 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.333815 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn" Jul 21 18:07:24.334006 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.333846 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af0b76eac3e4b52e1b613374ae2b6e94fe0246b9cfb6520ee155ad4ac1ca5caf" Jul 21 18:07:24.500762 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.500734 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" Jul 21 18:07:24.502822 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.502804 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" Jul 21 18:07:24.574729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.574695 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d401d4c-07a8-42c7-afeb-44ee744e346e-util\") pod \"0d401d4c-07a8-42c7-afeb-44ee744e346e\" (UID: \"0d401d4c-07a8-42c7-afeb-44ee744e346e\") " Jul 21 18:07:24.574889 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.574766 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-bundle\") pod \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\" (UID: \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\") " Jul 21 18:07:24.574889 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.574799 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26859\" (UniqueName: \"kubernetes.io/projected/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-kube-api-access-26859\") pod \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\" (UID: \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\") " Jul 21 18:07:24.574889 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.574827 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-util\") pod \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\" (UID: \"b1023893-fbc7-47d0-a023-a9f0a76b6ccd\") " Jul 21 18:07:24.574889 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.574854 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82rq9\" (UniqueName: \"kubernetes.io/projected/0d401d4c-07a8-42c7-afeb-44ee744e346e-kube-api-access-82rq9\") pod \"0d401d4c-07a8-42c7-afeb-44ee744e346e\" (UID: \"0d401d4c-07a8-42c7-afeb-44ee744e346e\") " Jul 21 18:07:24.574889 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.574873 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d401d4c-07a8-42c7-afeb-44ee744e346e-bundle\") pod \"0d401d4c-07a8-42c7-afeb-44ee744e346e\" (UID: \"0d401d4c-07a8-42c7-afeb-44ee744e346e\") " Jul 21 18:07:24.575383 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.575348 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-bundle" (OuterVolumeSpecName: "bundle") pod "b1023893-fbc7-47d0-a023-a9f0a76b6ccd" (UID: "b1023893-fbc7-47d0-a023-a9f0a76b6ccd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:07:24.575618 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.575588 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d401d4c-07a8-42c7-afeb-44ee744e346e-bundle" (OuterVolumeSpecName: "bundle") pod "0d401d4c-07a8-42c7-afeb-44ee744e346e" (UID: "0d401d4c-07a8-42c7-afeb-44ee744e346e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:07:24.577539 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.577516 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d401d4c-07a8-42c7-afeb-44ee744e346e-kube-api-access-82rq9" (OuterVolumeSpecName: "kube-api-access-82rq9") pod "0d401d4c-07a8-42c7-afeb-44ee744e346e" (UID: "0d401d4c-07a8-42c7-afeb-44ee744e346e"). InnerVolumeSpecName "kube-api-access-82rq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:07:24.577604 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.577547 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-kube-api-access-26859" (OuterVolumeSpecName: "kube-api-access-26859") pod "b1023893-fbc7-47d0-a023-a9f0a76b6ccd" (UID: "b1023893-fbc7-47d0-a023-a9f0a76b6ccd"). InnerVolumeSpecName "kube-api-access-26859". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:07:24.583652 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.583601 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-util" (OuterVolumeSpecName: "util") pod "b1023893-fbc7-47d0-a023-a9f0a76b6ccd" (UID: "b1023893-fbc7-47d0-a023-a9f0a76b6ccd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:07:24.583877 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.583859 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d401d4c-07a8-42c7-afeb-44ee744e346e-util" (OuterVolumeSpecName: "util") pod "0d401d4c-07a8-42c7-afeb-44ee744e346e" (UID: "0d401d4c-07a8-42c7-afeb-44ee744e346e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:07:24.675887 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.675848 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d401d4c-07a8-42c7-afeb-44ee744e346e-util\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:07:24.675887 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.675881 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-bundle\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:07:24.675887 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.675893 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26859\" (UniqueName: \"kubernetes.io/projected/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-kube-api-access-26859\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:07:24.676128 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.675907 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1023893-fbc7-47d0-a023-a9f0a76b6ccd-util\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:07:24.676128 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.675920 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82rq9\" (UniqueName: \"kubernetes.io/projected/0d401d4c-07a8-42c7-afeb-44ee744e346e-kube-api-access-82rq9\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:07:24.676128 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:24.675927 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d401d4c-07a8-42c7-afeb-44ee744e346e-bundle\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:07:25.340582 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:25.340554 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" Jul 21 18:07:25.341002 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:25.340551 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx" event={"ID":"0d401d4c-07a8-42c7-afeb-44ee744e346e","Type":"ContainerDied","Data":"b62a8106fecb690d0bac8f812c0743228faac63f768a46f24bceef3810f0a93c"} Jul 21 18:07:25.341002 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:25.340695 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b62a8106fecb690d0bac8f812c0743228faac63f768a46f24bceef3810f0a93c" Jul 21 18:07:25.342651 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:25.342622 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" event={"ID":"b1023893-fbc7-47d0-a023-a9f0a76b6ccd","Type":"ContainerDied","Data":"f2ec103340ca761e3b70d8ae689693e5fabe7573bd2868a43c1b68f77b73ab9d"} Jul 21 18:07:25.342793 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:25.342658 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz" Jul 21 18:07:25.342793 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:25.342669 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2ec103340ca761e3b70d8ae689693e5fabe7573bd2868a43c1b68f77b73ab9d" Jul 21 18:07:32.017340 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:32.017299 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:36.231758 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:36.231720 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:38.848118 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:38.848087 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:40.479402 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:40.479365 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:41.966534 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:41.966498 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:43.418243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:43.418207 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:43.686967 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:43.686815 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:44.023045 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:44.022964 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:46.251270 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:46.251235 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:48.437735 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:48.437695 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:48.544050 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:48.544012 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:50.439812 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:50.439766 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:50.541276 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:50.541231 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:53.093751 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:53.093714 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:53.224134 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:53.224092 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:55.628474 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:55.628438 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:07:55.731638 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:07:55.731601 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:03.291373 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291335 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb"] Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291636 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94cf9b0f-b8bc-466b-91af-2cb153917fb4" containerName="pull" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291646 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="94cf9b0f-b8bc-466b-91af-2cb153917fb4" containerName="pull" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291657 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f68f4f1-d977-4296-a62e-e9fda2a9fd59" containerName="pull" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291662 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f68f4f1-d977-4296-a62e-e9fda2a9fd59" containerName="pull" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291668 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1023893-fbc7-47d0-a023-a9f0a76b6ccd" containerName="pull" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291673 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1023893-fbc7-47d0-a023-a9f0a76b6ccd" containerName="pull" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291681 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f68f4f1-d977-4296-a62e-e9fda2a9fd59" containerName="extract" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291686 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f68f4f1-d977-4296-a62e-e9fda2a9fd59" containerName="extract" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291692 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d401d4c-07a8-42c7-afeb-44ee744e346e" containerName="util" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291697 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d401d4c-07a8-42c7-afeb-44ee744e346e" containerName="util" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291708 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94cf9b0f-b8bc-466b-91af-2cb153917fb4" containerName="util" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291714 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="94cf9b0f-b8bc-466b-91af-2cb153917fb4" containerName="util" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291720 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f68f4f1-d977-4296-a62e-e9fda2a9fd59" containerName="util" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291725 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f68f4f1-d977-4296-a62e-e9fda2a9fd59" containerName="util" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291731 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d401d4c-07a8-42c7-afeb-44ee744e346e" containerName="extract" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291736 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d401d4c-07a8-42c7-afeb-44ee744e346e" containerName="extract" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291743 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d401d4c-07a8-42c7-afeb-44ee744e346e" containerName="pull" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291748 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d401d4c-07a8-42c7-afeb-44ee744e346e" containerName="pull" Jul 21 18:08:03.291747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291755 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94cf9b0f-b8bc-466b-91af-2cb153917fb4" containerName="extract" Jul 21 18:08:03.292350 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291761 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="94cf9b0f-b8bc-466b-91af-2cb153917fb4" containerName="extract" Jul 21 18:08:03.292350 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291768 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1023893-fbc7-47d0-a023-a9f0a76b6ccd" containerName="extract" Jul 21 18:08:03.292350 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291773 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1023893-fbc7-47d0-a023-a9f0a76b6ccd" containerName="extract" Jul 21 18:08:03.292350 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291780 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1023893-fbc7-47d0-a023-a9f0a76b6ccd" containerName="util" Jul 21 18:08:03.292350 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291785 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1023893-fbc7-47d0-a023-a9f0a76b6ccd" containerName="util" Jul 21 18:08:03.292350 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291838 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1023893-fbc7-47d0-a023-a9f0a76b6ccd" containerName="extract" Jul 21 18:08:03.292350 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291846 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d401d4c-07a8-42c7-afeb-44ee744e346e" containerName="extract" Jul 21 18:08:03.292350 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291852 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f68f4f1-d977-4296-a62e-e9fda2a9fd59" containerName="extract" Jul 21 18:08:03.292350 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.291858 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="94cf9b0f-b8bc-466b-91af-2cb153917fb4" containerName="extract" Jul 21 18:08:03.294643 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.294625 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:03.297354 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.297328 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 18:08:03.297637 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.297616 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v7plk\"" Jul 21 18:08:03.297727 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.297652 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 18:08:03.309971 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.309910 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb"] Jul 21 18:08:03.416387 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.416354 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ecca4d0-0a84-41c3-a7db-762961f03084-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-lcjkb\" (UID: \"6ecca4d0-0a84-41c3-a7db-762961f03084\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:03.416387 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.416396 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsmsp\" (UniqueName: \"kubernetes.io/projected/6ecca4d0-0a84-41c3-a7db-762961f03084-kube-api-access-dsmsp\") pod \"kuadrant-operator-controller-manager-55467d9f7-lcjkb\" (UID: \"6ecca4d0-0a84-41c3-a7db-762961f03084\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:03.517004 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.516960 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ecca4d0-0a84-41c3-a7db-762961f03084-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-lcjkb\" (UID: \"6ecca4d0-0a84-41c3-a7db-762961f03084\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:03.517169 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.517032 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsmsp\" (UniqueName: \"kubernetes.io/projected/6ecca4d0-0a84-41c3-a7db-762961f03084-kube-api-access-dsmsp\") pod \"kuadrant-operator-controller-manager-55467d9f7-lcjkb\" (UID: \"6ecca4d0-0a84-41c3-a7db-762961f03084\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:03.517350 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.517330 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ecca4d0-0a84-41c3-a7db-762961f03084-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-lcjkb\" (UID: \"6ecca4d0-0a84-41c3-a7db-762961f03084\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:03.537679 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.537646 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsmsp\" (UniqueName: \"kubernetes.io/projected/6ecca4d0-0a84-41c3-a7db-762961f03084-kube-api-access-dsmsp\") pod \"kuadrant-operator-controller-manager-55467d9f7-lcjkb\" (UID: \"6ecca4d0-0a84-41c3-a7db-762961f03084\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:03.604838 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.604798 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:03.710288 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.710175 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb"] Jul 21 18:08:03.713653 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.713620 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb"] Jul 21 18:08:03.750122 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.750093 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2"] Jul 21 18:08:03.753726 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.753698 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" Jul 21 18:08:03.798301 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.798265 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2"] Jul 21 18:08:03.819531 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.819497 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pgqr\" (UniqueName: \"kubernetes.io/projected/5e823f38-4d30-4e7a-b320-d0d6f6416b10-kube-api-access-5pgqr\") pod \"kuadrant-operator-controller-manager-55467d9f7-x69t2\" (UID: \"5e823f38-4d30-4e7a-b320-d0d6f6416b10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" Jul 21 18:08:03.819725 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.819547 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e823f38-4d30-4e7a-b320-d0d6f6416b10-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-x69t2\" (UID: \"5e823f38-4d30-4e7a-b320-d0d6f6416b10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" Jul 21 18:08:03.894131 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:08:03.894095 2568 log.go:32] "RunPodSandbox from runtime service failed" err=< Jul 21 18:08:03.894131 ip-10-0-136-17 kubenswrapper[2568]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-lcjkb_kuadrant-system_6ecca4d0-0a84-41c3-a7db-762961f03084_0(763b35059dd5ca41b1ef0045ba46631c6970be121ed1f0171ccbfaa0f793073d): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-lcjkb to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"763b35059dd5ca41b1ef0045ba46631c6970be121ed1f0171ccbfaa0f793073d" Netns:"/var/run/netns/9b09d3a6-9cb1-4356-b386-321fbc056211" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-lcjkb;K8S_POD_INFRA_CONTAINER_ID=763b35059dd5ca41b1ef0045ba46631c6970be121ed1f0171ccbfaa0f793073d;K8S_POD_UID=6ecca4d0-0a84-41c3-a7db-762961f03084" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb/6ecca4d0-0a84-41c3-a7db-762961f03084:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb 763b35059dd5ca41b1ef0045ba46631c6970be121ed1f0171ccbfaa0f793073d network default NAD default NAD key default] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb 763b35059dd5ca41b1ef0045ba46631c6970be121ed1f0171ccbfaa0f793073d network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:84:00:27 [10.132.0.39/23] Jul 21 18:08:03.894131 ip-10-0-136-17 kubenswrapper[2568]: ' Jul 21 18:08:03.894131 ip-10-0-136-17 kubenswrapper[2568]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 21 18:08:03.894131 ip-10-0-136-17 kubenswrapper[2568]: > Jul 21 18:08:03.894335 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:08:03.894161 2568 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=< Jul 21 18:08:03.894335 ip-10-0-136-17 kubenswrapper[2568]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-lcjkb_kuadrant-system_6ecca4d0-0a84-41c3-a7db-762961f03084_0(763b35059dd5ca41b1ef0045ba46631c6970be121ed1f0171ccbfaa0f793073d): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-lcjkb to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"763b35059dd5ca41b1ef0045ba46631c6970be121ed1f0171ccbfaa0f793073d" Netns:"/var/run/netns/9b09d3a6-9cb1-4356-b386-321fbc056211" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-lcjkb;K8S_POD_INFRA_CONTAINER_ID=763b35059dd5ca41b1ef0045ba46631c6970be121ed1f0171ccbfaa0f793073d;K8S_POD_UID=6ecca4d0-0a84-41c3-a7db-762961f03084" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb/6ecca4d0-0a84-41c3-a7db-762961f03084:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb 763b35059dd5ca41b1ef0045ba46631c6970be121ed1f0171ccbfaa0f793073d network default NAD default NAD key default] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb 763b35059dd5ca41b1ef0045ba46631c6970be121ed1f0171ccbfaa0f793073d network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:84:00:27 [10.132.0.39/23] Jul 21 18:08:03.894335 ip-10-0-136-17 kubenswrapper[2568]: ' Jul 21 18:08:03.894335 ip-10-0-136-17 kubenswrapper[2568]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 21 18:08:03.894335 ip-10-0-136-17 kubenswrapper[2568]: > pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:03.920550 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.920509 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e823f38-4d30-4e7a-b320-d0d6f6416b10-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-x69t2\" (UID: \"5e823f38-4d30-4e7a-b320-d0d6f6416b10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" Jul 21 18:08:03.920689 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.920603 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pgqr\" (UniqueName: \"kubernetes.io/projected/5e823f38-4d30-4e7a-b320-d0d6f6416b10-kube-api-access-5pgqr\") pod \"kuadrant-operator-controller-manager-55467d9f7-x69t2\" (UID: \"5e823f38-4d30-4e7a-b320-d0d6f6416b10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" Jul 21 18:08:03.920910 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.920887 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e823f38-4d30-4e7a-b320-d0d6f6416b10-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-x69t2\" (UID: \"5e823f38-4d30-4e7a-b320-d0d6f6416b10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" Jul 21 18:08:03.935299 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.935273 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pgqr\" (UniqueName: \"kubernetes.io/projected/5e823f38-4d30-4e7a-b320-d0d6f6416b10-kube-api-access-5pgqr\") pod \"kuadrant-operator-controller-manager-55467d9f7-x69t2\" (UID: \"5e823f38-4d30-4e7a-b320-d0d6f6416b10\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" Jul 21 18:08:03.967751 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:03.967711 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:04.064086 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:04.064051 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" Jul 21 18:08:04.066037 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:04.066011 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:04.165379 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:04.165302 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:04.218427 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:08:04.218398 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e823f38_4d30_4e7a_b320_d0d6f6416b10.slice/crio-ab1cfbc1648a8c591cd7b9562c2b974d13d215405cf0e31187615221ee1b38ad WatchSource:0}: Error finding container ab1cfbc1648a8c591cd7b9562c2b974d13d215405cf0e31187615221ee1b38ad: Status 404 returned error can't find the container with id ab1cfbc1648a8c591cd7b9562c2b974d13d215405cf0e31187615221ee1b38ad Jul 21 18:08:04.218753 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:04.218733 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2"] Jul 21 18:08:04.546209 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:04.546111 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" event={"ID":"5e823f38-4d30-4e7a-b320-d0d6f6416b10","Type":"ContainerStarted","Data":"ab1cfbc1648a8c591cd7b9562c2b974d13d215405cf0e31187615221ee1b38ad"} Jul 21 18:08:04.546209 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:04.546152 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:04.548447 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:08:04.548415 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lcjkb\" is forbidden: User \"system:node:ip-10-0-136-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-17.ec2.internal' and this object" podUID="6ecca4d0-0a84-41c3-a7db-762961f03084" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:04.554276 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:04.554257 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:04.556567 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:08:04.556544 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lcjkb\" is forbidden: User \"system:node:ip-10-0-136-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-17.ec2.internal' and this object" podUID="6ecca4d0-0a84-41c3-a7db-762961f03084" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:04.627779 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:04.627739 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsmsp\" (UniqueName: \"kubernetes.io/projected/6ecca4d0-0a84-41c3-a7db-762961f03084-kube-api-access-dsmsp\") pod \"6ecca4d0-0a84-41c3-a7db-762961f03084\" (UID: \"6ecca4d0-0a84-41c3-a7db-762961f03084\") " Jul 21 18:08:04.627779 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:04.627784 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ecca4d0-0a84-41c3-a7db-762961f03084-extensions-socket-volume\") pod \"6ecca4d0-0a84-41c3-a7db-762961f03084\" (UID: \"6ecca4d0-0a84-41c3-a7db-762961f03084\") " Jul 21 18:08:04.628106 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:04.628082 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ecca4d0-0a84-41c3-a7db-762961f03084-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6ecca4d0-0a84-41c3-a7db-762961f03084" (UID: "6ecca4d0-0a84-41c3-a7db-762961f03084"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:08:04.629876 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:04.629856 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ecca4d0-0a84-41c3-a7db-762961f03084-kube-api-access-dsmsp" (OuterVolumeSpecName: "kube-api-access-dsmsp") pod "6ecca4d0-0a84-41c3-a7db-762961f03084" (UID: "6ecca4d0-0a84-41c3-a7db-762961f03084"). InnerVolumeSpecName "kube-api-access-dsmsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:08:04.729332 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:04.729294 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsmsp\" (UniqueName: \"kubernetes.io/projected/6ecca4d0-0a84-41c3-a7db-762961f03084-kube-api-access-dsmsp\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:08:04.729332 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:04.729325 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ecca4d0-0a84-41c3-a7db-762961f03084-extensions-socket-volume\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:08:05.220174 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:05.220141 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ecca4d0-0a84-41c3-a7db-762961f03084" path="/var/lib/kubelet/pods/6ecca4d0-0a84-41c3-a7db-762961f03084/volumes" Jul 21 18:08:05.551709 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:05.551638 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:05.556821 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:08:05.556777 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lcjkb\" is forbidden: User \"system:node:ip-10-0-136-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-17.ec2.internal' and this object" podUID="6ecca4d0-0a84-41c3-a7db-762961f03084" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lcjkb" Jul 21 18:08:06.475695 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:06.475647 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:06.576237 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:06.576194 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:07.513968 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:07.513870 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:07.612361 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:07.612313 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:09.475593 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:09.475559 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:09.575342 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:09.575252 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" event={"ID":"5e823f38-4d30-4e7a-b320-d0d6f6416b10","Type":"ContainerStarted","Data":"f53917d91a43e801593d7da0746170a9c7c8f38e0fffc2a9a337bdfb837716d0"} Jul 21 18:08:09.575491 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:09.575346 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" Jul 21 18:08:09.584652 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:09.584623 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:09.615040 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:09.614976 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" podStartSLOduration=1.5338779470000001 podStartE2EDuration="6.614956196s" totalImagesPullingTime="5.081078249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:08:09.614390718 +0000 UTC m=+490.882702161" watchObservedRunningTime="2026-07-21 18:08:09.614956196 +0000 UTC m=+490.883267634" Jul 21 18:08:13.265479 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:13.265440 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:13.368373 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:13.368336 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:14.489785 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:14.489734 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:20.582976 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:20.582924 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" Jul 21 18:08:21.915244 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:21.915208 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:22.017144 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:22.017103 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:37.139622 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:37.139583 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:37.526613 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:37.526527 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:38.757374 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:38.757339 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2"] Jul 21 18:08:38.757882 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:38.757546 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" podUID="5e823f38-4d30-4e7a-b320-d0d6f6416b10" containerName="manager" containerID="cri-o://f53917d91a43e801593d7da0746170a9c7c8f38e0fffc2a9a337bdfb837716d0" gracePeriod=10 Jul 21 18:08:39.018699 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.018635 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" Jul 21 18:08:39.025123 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.025097 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pgqr\" (UniqueName: \"kubernetes.io/projected/5e823f38-4d30-4e7a-b320-d0d6f6416b10-kube-api-access-5pgqr\") pod \"5e823f38-4d30-4e7a-b320-d0d6f6416b10\" (UID: \"5e823f38-4d30-4e7a-b320-d0d6f6416b10\") " Jul 21 18:08:39.025238 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.025161 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e823f38-4d30-4e7a-b320-d0d6f6416b10-extensions-socket-volume\") pod \"5e823f38-4d30-4e7a-b320-d0d6f6416b10\" (UID: \"5e823f38-4d30-4e7a-b320-d0d6f6416b10\") " Jul 21 18:08:39.025669 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.025642 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e823f38-4d30-4e7a-b320-d0d6f6416b10-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5e823f38-4d30-4e7a-b320-d0d6f6416b10" (UID: "5e823f38-4d30-4e7a-b320-d0d6f6416b10"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 18:08:39.027187 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.027165 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e823f38-4d30-4e7a-b320-d0d6f6416b10-kube-api-access-5pgqr" (OuterVolumeSpecName: "kube-api-access-5pgqr") pod "5e823f38-4d30-4e7a-b320-d0d6f6416b10" (UID: "5e823f38-4d30-4e7a-b320-d0d6f6416b10"). InnerVolumeSpecName "kube-api-access-5pgqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:08:39.125601 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.125567 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e823f38-4d30-4e7a-b320-d0d6f6416b10-extensions-socket-volume\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:08:39.125601 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.125595 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pgqr\" (UniqueName: \"kubernetes.io/projected/5e823f38-4d30-4e7a-b320-d0d6f6416b10-kube-api-access-5pgqr\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:08:39.725190 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.725153 2568 generic.go:356] "Generic (PLEG): container finished" podID="5e823f38-4d30-4e7a-b320-d0d6f6416b10" containerID="f53917d91a43e801593d7da0746170a9c7c8f38e0fffc2a9a337bdfb837716d0" exitCode=0 Jul 21 18:08:39.725378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.725195 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" Jul 21 18:08:39.725378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.725237 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" event={"ID":"5e823f38-4d30-4e7a-b320-d0d6f6416b10","Type":"ContainerDied","Data":"f53917d91a43e801593d7da0746170a9c7c8f38e0fffc2a9a337bdfb837716d0"} Jul 21 18:08:39.725378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.725293 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2" event={"ID":"5e823f38-4d30-4e7a-b320-d0d6f6416b10","Type":"ContainerDied","Data":"ab1cfbc1648a8c591cd7b9562c2b974d13d215405cf0e31187615221ee1b38ad"} Jul 21 18:08:39.725378 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.725319 2568 scope.go:117] "RemoveContainer" containerID="f53917d91a43e801593d7da0746170a9c7c8f38e0fffc2a9a337bdfb837716d0" Jul 21 18:08:39.734821 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.734799 2568 scope.go:117] "RemoveContainer" containerID="f53917d91a43e801593d7da0746170a9c7c8f38e0fffc2a9a337bdfb837716d0" Jul 21 18:08:39.735166 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:08:39.735141 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f53917d91a43e801593d7da0746170a9c7c8f38e0fffc2a9a337bdfb837716d0\": container with ID starting with f53917d91a43e801593d7da0746170a9c7c8f38e0fffc2a9a337bdfb837716d0 not found: ID does not exist" containerID="f53917d91a43e801593d7da0746170a9c7c8f38e0fffc2a9a337bdfb837716d0" Jul 21 18:08:39.735223 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.735180 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f53917d91a43e801593d7da0746170a9c7c8f38e0fffc2a9a337bdfb837716d0"} err="failed to get container status \"f53917d91a43e801593d7da0746170a9c7c8f38e0fffc2a9a337bdfb837716d0\": rpc error: code = NotFound desc = could not find container \"f53917d91a43e801593d7da0746170a9c7c8f38e0fffc2a9a337bdfb837716d0\": container with ID starting with f53917d91a43e801593d7da0746170a9c7c8f38e0fffc2a9a337bdfb837716d0 not found: ID does not exist" Jul 21 18:08:39.743685 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.743650 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2"] Jul 21 18:08:39.749295 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:39.749270 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-x69t2"] Jul 21 18:08:41.219078 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:41.219041 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e823f38-4d30-4e7a-b320-d0d6f6416b10" path="/var/lib/kubelet/pods/5e823f38-4d30-4e7a-b320-d0d6f6416b10/volumes" Jul 21 18:08:56.840544 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:56.840452 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:56.960813 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:56.960773 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:57.066801 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.066764 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:57.277206 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.277167 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:08:57.277594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.277576 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e823f38-4d30-4e7a-b320-d0d6f6416b10" containerName="manager" Jul 21 18:08:57.277682 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.277597 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e823f38-4d30-4e7a-b320-d0d6f6416b10" containerName="manager" Jul 21 18:08:57.277739 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.277685 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e823f38-4d30-4e7a-b320-d0d6f6416b10" containerName="manager" Jul 21 18:08:57.280510 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.280488 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-49bb6" Jul 21 18:08:57.282733 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.282712 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 21 18:08:57.282989 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.282972 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 18:08:57.283597 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.283583 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xcwkl\"" Jul 21 18:08:57.283652 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.283581 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 18:08:57.290188 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.290167 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:08:57.326693 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.326645 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:08:57.378591 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.378550 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8lx4\" (UniqueName: \"kubernetes.io/projected/c576b06f-1bc5-4c83-b25a-45757fbbf3d5-kube-api-access-j8lx4\") pod \"limitador-limitador-54d6d988d5-49bb6\" (UID: \"c576b06f-1bc5-4c83-b25a-45757fbbf3d5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-49bb6" Jul 21 18:08:57.378769 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.378644 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c576b06f-1bc5-4c83-b25a-45757fbbf3d5-config-file\") pod \"limitador-limitador-54d6d988d5-49bb6\" (UID: \"c576b06f-1bc5-4c83-b25a-45757fbbf3d5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-49bb6" Jul 21 18:08:57.479786 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.479750 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c576b06f-1bc5-4c83-b25a-45757fbbf3d5-config-file\") pod \"limitador-limitador-54d6d988d5-49bb6\" (UID: \"c576b06f-1bc5-4c83-b25a-45757fbbf3d5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-49bb6" Jul 21 18:08:57.480017 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.479823 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8lx4\" (UniqueName: \"kubernetes.io/projected/c576b06f-1bc5-4c83-b25a-45757fbbf3d5-kube-api-access-j8lx4\") pod \"limitador-limitador-54d6d988d5-49bb6\" (UID: \"c576b06f-1bc5-4c83-b25a-45757fbbf3d5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-49bb6" Jul 21 18:08:57.480440 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.480417 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c576b06f-1bc5-4c83-b25a-45757fbbf3d5-config-file\") pod \"limitador-limitador-54d6d988d5-49bb6\" (UID: \"c576b06f-1bc5-4c83-b25a-45757fbbf3d5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-49bb6" Jul 21 18:08:57.488245 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.488222 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8lx4\" (UniqueName: \"kubernetes.io/projected/c576b06f-1bc5-4c83-b25a-45757fbbf3d5-kube-api-access-j8lx4\") pod \"limitador-limitador-54d6d988d5-49bb6\" (UID: \"c576b06f-1bc5-4c83-b25a-45757fbbf3d5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-49bb6" Jul 21 18:08:57.591688 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.591651 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-49bb6" Jul 21 18:08:57.713697 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.713666 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:57.730828 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.730717 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:08:57.733114 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:08:57.733081 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc576b06f_1bc5_4c83_b25a_45757fbbf3d5.slice/crio-870eb32cace99f8c2e449b6905c67afd746445a2446f79c61643a347be7bd94e WatchSource:0}: Error finding container 870eb32cace99f8c2e449b6905c67afd746445a2446f79c61643a347be7bd94e: Status 404 returned error can't find the container with id 870eb32cace99f8c2e449b6905c67afd746445a2446f79c61643a347be7bd94e Jul 21 18:08:57.777591 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.777549 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:08:57.816620 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:08:57.816584 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-49bb6" event={"ID":"c576b06f-1bc5-4c83-b25a-45757fbbf3d5","Type":"ContainerStarted","Data":"870eb32cace99f8c2e449b6905c67afd746445a2446f79c61643a347be7bd94e"} Jul 21 18:09:00.835085 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:00.835049 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-49bb6" event={"ID":"c576b06f-1bc5-4c83-b25a-45757fbbf3d5","Type":"ContainerStarted","Data":"d5c8db1693eb532cd6ccd64cf98c161e0c6d6fb3e25922ae7d548ef6739b4690"} Jul 21 18:09:00.835473 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:00.835116 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-49bb6" Jul 21 18:09:00.853430 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:00.853377 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-49bb6" podStartSLOduration=1.406856188 podStartE2EDuration="3.853361131s" totalImagesPullingTime="2.446504943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:08:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:09:00.851174609 +0000 UTC m=+542.119486052" watchObservedRunningTime="2026-07-21 18:09:00.853361131 +0000 UTC m=+542.121672573" Jul 21 18:09:08.700844 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:08.700804 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:11.840699 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:11.840665 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-49bb6" Jul 21 18:09:15.727290 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:15.727255 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:15.826206 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:15.826168 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:16.217604 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:16.217566 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:18.599065 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:18.599028 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:27.537519 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:27.537483 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:28.616408 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:28.616374 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:28.908595 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:28.908500 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:35.049566 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:35.049524 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:35.151459 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:35.151400 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:35.216712 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:35.216674 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:35.401605 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:35.401565 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:38.595696 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:38.595658 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:38.697584 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:38.697550 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:39.656653 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:39.656618 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:09:59.118030 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:59.118001 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:09:59.118564 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:59.118545 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:09:59.124838 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:59.124804 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:09:59.125241 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:09:59.125222 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:10:04.262660 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:04.262623 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:10:06.088120 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:06.088083 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:10:06.145275 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:06.145225 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:10:09.922869 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:09.922822 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:10:10.017351 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:10.017318 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:10:10.484389 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:10.484346 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:10:10.981672 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:10.981640 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:10:11.085779 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:11.085743 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:10:11.185561 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:11.185524 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:10:11.190061 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:11.190031 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:10:20.558416 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:20.558379 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:10:20.639285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:20.639244 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:10:39.053075 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:39.052963 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:10:39.532887 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:39.532844 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:10:43.650841 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:43.650804 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:10:44.425289 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.425257 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-87w97"] Jul 21 18:10:44.428655 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.428635 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.432068 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.432047 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-g5lsg\"" Jul 21 18:10:44.432794 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.432773 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 21 18:10:44.432893 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.432860 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 21 18:10:44.432975 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.432956 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 21 18:10:44.445827 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.445807 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-87w97"] Jul 21 18:10:44.487287 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.487251 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:10:44.495275 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.495248 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85fc0cee-7513-4376-a953-02787f9e096f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.495417 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.495291 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85fc0cee-7513-4376-a953-02787f9e096f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.495417 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.495313 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85fc0cee-7513-4376-a953-02787f9e096f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.495417 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.495363 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtm5t\" (UniqueName: \"kubernetes.io/projected/85fc0cee-7513-4376-a953-02787f9e096f-kube-api-access-rtm5t\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.495417 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.495391 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85fc0cee-7513-4376-a953-02787f9e096f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.495607 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.495467 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85fc0cee-7513-4376-a953-02787f9e096f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.596176 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.596135 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85fc0cee-7513-4376-a953-02787f9e096f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.596370 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.596195 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85fc0cee-7513-4376-a953-02787f9e096f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.596370 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.596224 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85fc0cee-7513-4376-a953-02787f9e096f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.596370 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.596260 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtm5t\" (UniqueName: \"kubernetes.io/projected/85fc0cee-7513-4376-a953-02787f9e096f-kube-api-access-rtm5t\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.596370 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.596293 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85fc0cee-7513-4376-a953-02787f9e096f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.596370 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.596322 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85fc0cee-7513-4376-a953-02787f9e096f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.596694 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.596668 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85fc0cee-7513-4376-a953-02787f9e096f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.596694 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.596686 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85fc0cee-7513-4376-a953-02787f9e096f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.596810 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.596723 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85fc0cee-7513-4376-a953-02787f9e096f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.598563 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.598535 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85fc0cee-7513-4376-a953-02787f9e096f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.598700 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.598685 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85fc0cee-7513-4376-a953-02787f9e096f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.609342 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.609316 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtm5t\" (UniqueName: \"kubernetes.io/projected/85fc0cee-7513-4376-a953-02787f9e096f-kube-api-access-rtm5t\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87w97\" (UID: \"85fc0cee-7513-4376-a953-02787f9e096f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.739285 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.739192 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:44.887407 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.887378 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-87w97"] Jul 21 18:10:44.889809 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:10:44.889784 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85fc0cee_7513_4376_a953_02787f9e096f.slice/crio-abcd6e0b9c9112bb2371d461dd520198e3c0fadfb738d69d4cbfcc709c0f1c1b WatchSource:0}: Error finding container abcd6e0b9c9112bb2371d461dd520198e3c0fadfb738d69d4cbfcc709c0f1c1b: Status 404 returned error can't find the container with id abcd6e0b9c9112bb2371d461dd520198e3c0fadfb738d69d4cbfcc709c0f1c1b Jul 21 18:10:44.890975 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:44.890956 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 18:10:45.233076 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:45.233041 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:10:45.356549 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:45.356498 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" event={"ID":"85fc0cee-7513-4376-a953-02787f9e096f","Type":"ContainerStarted","Data":"abcd6e0b9c9112bb2371d461dd520198e3c0fadfb738d69d4cbfcc709c0f1c1b"} Jul 21 18:10:46.367302 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:46.367268 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" event={"ID":"85fc0cee-7513-4376-a953-02787f9e096f","Type":"ContainerStarted","Data":"e8ac5199bcf7d1b52970550a0fe5c3b4a506dd5b7b550cc18e0195cb640ef44b"} Jul 21 18:10:46.389752 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:46.389702 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" podStartSLOduration=1.3237786919999999 podStartE2EDuration="2.389685438s" totalImagesPullingTime="1.065906746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:10:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:10:46.38855172 +0000 UTC m=+647.656863160" watchObservedRunningTime="2026-07-21 18:10:46.389685438 +0000 UTC m=+647.657996878" Jul 21 18:10:54.740377 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:54.740325 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:54.740777 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:54.740407 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:54.752917 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:54.752895 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:10:55.421844 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:10:55.421815 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87w97" Jul 21 18:11:06.286805 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:11:06.286772 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:11:07.693243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:11:07.693207 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:11:09.079590 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:11:09.079547 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:11:14.146005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:11:14.145966 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:11:15.053520 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:11:15.053485 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:11:23.437630 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:11:23.437594 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:11:34.902769 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:11:34.902729 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:11:37.250391 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:11:37.250352 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:12:26.754055 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:12:26.754013 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:12:26.854494 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:12:26.854450 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:12:30.512116 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:12:30.512072 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:12:30.609316 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:12:30.609276 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:12:31.703091 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:12:31.703055 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:12:31.810706 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:12:31.810667 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:12:59.730230 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:12:59.730187 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:13:10.241055 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:13:10.241016 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:13:21.432511 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:13:21.432433 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:13:43.316466 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:13:43.316432 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:13:43.371312 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:13:43.371278 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:14:36.203984 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:14:36.203947 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:14:37.074870 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:14:37.074825 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:14:37.176648 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:14:37.176611 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:14:37.464279 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:14:37.464234 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:14:59.157365 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:14:59.157329 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:14:59.158022 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:14:59.158000 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:14:59.162989 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:14:59.162966 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:14:59.163612 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:14:59.163594 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:15:00.136773 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:00.136741 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744295-9rh92"] Jul 21 18:15:00.140088 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:00.140070 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744295-9rh92" Jul 21 18:15:00.142298 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:00.142281 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:15:00.147700 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:00.147252 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744295-9rh92"] Jul 21 18:15:00.151658 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:00.151638 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744295-9rh92" Jul 21 18:15:00.285496 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:00.285420 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744295-9rh92"] Jul 21 18:15:00.288131 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:15:00.288104 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27af902b_18bc_49f4_9721_a481d4ae273f.slice/crio-a8b4132d9f05a0e382a2d2485b5316dbd2cc190f76fdb8e4828ce60a242731c2 WatchSource:0}: Error finding container a8b4132d9f05a0e382a2d2485b5316dbd2cc190f76fdb8e4828ce60a242731c2: Status 404 returned error can't find the container with id a8b4132d9f05a0e382a2d2485b5316dbd2cc190f76fdb8e4828ce60a242731c2 Jul 21 18:15:00.634120 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:00.634086 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744295-9rh92" event={"ID":"27af902b-18bc-49f4-9721-a481d4ae273f","Type":"ContainerStarted","Data":"a8b4132d9f05a0e382a2d2485b5316dbd2cc190f76fdb8e4828ce60a242731c2"} Jul 21 18:15:01.036859 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:01.036774 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:15:02.654010 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:02.653972 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744295-9rh92" event={"ID":"27af902b-18bc-49f4-9721-a481d4ae273f","Type":"ContainerStarted","Data":"e1f701564285f697fbc9b3f701810d8517f52089aa991613b96dfad78fa9181d"} Jul 21 18:15:02.671032 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:02.670970 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744295-9rh92" podStartSLOduration=0.722764668 podStartE2EDuration="2.670931143s" totalImagesPullingTime="1.948166475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:15:02.669968245 +0000 UTC m=+903.938279677" watchObservedRunningTime="2026-07-21 18:15:02.670931143 +0000 UTC m=+903.939242584" Jul 21 18:15:03.661146 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:03.661105 2568 generic.go:356] "Generic (PLEG): container finished" podID="27af902b-18bc-49f4-9721-a481d4ae273f" containerID="e1f701564285f697fbc9b3f701810d8517f52089aa991613b96dfad78fa9181d" exitCode=0 Jul 21 18:15:03.661511 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:03.661185 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744295-9rh92" event={"ID":"27af902b-18bc-49f4-9721-a481d4ae273f","Type":"ContainerDied","Data":"e1f701564285f697fbc9b3f701810d8517f52089aa991613b96dfad78fa9181d"} Jul 21 18:15:04.796033 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:04.796004 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744295-9rh92" Jul 21 18:15:05.672045 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:05.672013 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744295-9rh92" event={"ID":"27af902b-18bc-49f4-9721-a481d4ae273f","Type":"ContainerDied","Data":"a8b4132d9f05a0e382a2d2485b5316dbd2cc190f76fdb8e4828ce60a242731c2"} Jul 21 18:15:05.672235 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:05.672068 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8b4132d9f05a0e382a2d2485b5316dbd2cc190f76fdb8e4828ce60a242731c2" Jul 21 18:15:05.672235 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:05.672026 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744295-9rh92" Jul 21 18:15:14.170495 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:14.170455 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:15:14.262762 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:14.262730 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:15:14.391529 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:14.391480 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:15:21.854777 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:21.854740 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:15:21.961241 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:21.961200 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:15:22.053300 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:22.053270 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:15:22.194254 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:22.194156 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:15:28.013206 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:28.013159 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:15:28.084692 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:28.084658 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:15:41.954901 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:41.954861 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:15:45.428754 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:45.428711 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:15:54.536876 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:15:54.536829 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:16:03.144798 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:16:03.144762 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:16:12.330687 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:16:12.330646 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:16:23.644717 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:16:23.644632 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:16:33.237571 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:16:33.237530 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:16:45.739785 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:16:45.739747 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:16:49.410906 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:16:49.410868 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:16:56.088005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:16:56.087969 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:17:20.935166 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:17:20.935132 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:17:32.468181 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:17:32.468145 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:17:55.845849 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:17:55.845766 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:18:06.734663 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:18:06.734623 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:18:22.932571 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:18:22.932533 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:18:32.338721 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:18:32.338674 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:18:42.432163 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:18:42.432126 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:18:52.040044 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:18:52.040007 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:19:02.179155 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:19:02.179122 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:19:20.276289 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:19:20.276256 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:19:59.199418 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:19:59.199391 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:19:59.199976 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:19:59.199391 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:19:59.205257 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:19:59.205227 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:19:59.205393 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:19:59.205375 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:20:50.632033 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:20:50.631994 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:20:59.626289 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:20:59.626202 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:21:13.329750 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:21:13.329712 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:21:22.623050 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:21:22.623008 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:21:31.736029 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:21:31.735985 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:21:42.031769 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:21:42.031736 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:21:51.830814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:21:51.830771 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:22:03.132457 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:22:03.132420 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:22:11.535946 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:22:11.535908 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:22:22.536237 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:22:22.536155 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:22:30.130992 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:22:30.130950 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:22:42.230321 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:22:42.230288 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:22:45.236186 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:22:45.236148 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:23:04.434377 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:23:04.434338 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:23:13.930030 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:23:13.929992 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:23:17.462221 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:23:17.462190 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:23:29.137161 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:23:29.137126 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:23:30.968520 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:23:30.968486 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:23:42.535435 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:23:42.535395 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:23:49.013947 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:23:49.013903 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:23:50.329898 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:23:50.329856 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:23:51.729624 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:23:51.729547 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:24:06.829254 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:24:06.829216 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:24:14.035350 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:24:14.035315 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:24:23.535101 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:24:23.535062 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:24:31.237794 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:24:31.237753 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:24:40.436860 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:24:40.436819 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:24:50.732642 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:24:50.732608 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:24:59.242100 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:24:59.242068 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:24:59.243051 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:24:59.243030 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:24:59.248346 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:24:59.248319 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:24:59.249143 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:24:59.249124 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:25:01.127760 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:25:01.127718 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:25:10.633155 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:25:10.633118 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:25:12.464280 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:25:12.464247 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:25:20.427979 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:25:20.427930 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:25:30.633463 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:25:30.633379 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:25:42.241501 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:25:42.241468 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:25:52.038181 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:25:52.038143 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:26:01.426620 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:26:01.426585 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:26:14.438513 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:26:14.438475 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:26:22.939809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:26:22.939769 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:26:35.719284 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:26:35.719246 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:26:48.335898 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:26:48.335860 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:27:01.528151 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:01.528065 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:27:03.781796 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:03.781751 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:15.635093 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:15.635060 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:27:24.933548 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:24.933511 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:27:25.200218 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:25.200132 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:28.157800 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:28.157763 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:28.255250 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:28.255214 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:28.604111 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:28.604051 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:34.494580 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:34.494545 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:34.593986 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:34.593950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:34.696633 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:34.696597 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:34.864644 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:34.864607 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:27:35.026442 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:35.026408 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:35.583701 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:35.583671 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:37.646219 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:37.646180 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:37.729059 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:37.728993 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:37.821072 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:37.821041 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:37.944015 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:37.943918 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:38.448132 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:38.448098 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:38.549294 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:38.549263 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:27:44.232469 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:44.232432 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:27:53.375877 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:27:53.375842 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:03.180727 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:03.180691 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b"] Jul 21 18:28:03.181274 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:03.181047 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27af902b-18bc-49f4-9721-a481d4ae273f" containerName="cleanup" Jul 21 18:28:03.181274 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:03.181060 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="27af902b-18bc-49f4-9721-a481d4ae273f" containerName="cleanup" Jul 21 18:28:03.181274 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:03.181129 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="27af902b-18bc-49f4-9721-a481d4ae273f" containerName="cleanup" Jul 21 18:28:03.184053 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:03.184034 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b" Jul 21 18:28:03.186242 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:03.186220 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 18:28:03.186350 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:03.186274 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:28:03.190354 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:03.190328 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b"] Jul 21 18:28:03.266813 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:03.266767 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d994b2fb-347a-4246-96b4-bcbb8c24f92b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b\" (UID: \"d994b2fb-347a-4246-96b4-bcbb8c24f92b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b" Jul 21 18:28:03.367457 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:03.367421 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d994b2fb-347a-4246-96b4-bcbb8c24f92b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b\" (UID: \"d994b2fb-347a-4246-96b4-bcbb8c24f92b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b" Jul 21 18:28:03.368135 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:03.368109 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d994b2fb-347a-4246-96b4-bcbb8c24f92b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b\" (UID: \"d994b2fb-347a-4246-96b4-bcbb8c24f92b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b" Jul 21 18:28:03.496232 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:03.496134 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b" Jul 21 18:28:03.626291 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:03.626261 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b"] Jul 21 18:28:03.628333 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:28:03.628308 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd994b2fb_347a_4246_96b4_bcbb8c24f92b.slice/crio-9fcbc1e00e44a82b16e21b10f09dbc74140c5cd8c7b18af8bc13d820293d48bd WatchSource:0}: Error finding container 9fcbc1e00e44a82b16e21b10f09dbc74140c5cd8c7b18af8bc13d820293d48bd: Status 404 returned error can't find the container with id 9fcbc1e00e44a82b16e21b10f09dbc74140c5cd8c7b18af8bc13d820293d48bd Jul 21 18:28:04.607863 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:04.607828 2568 generic.go:356] "Generic (PLEG): container finished" podID="d994b2fb-347a-4246-96b4-bcbb8c24f92b" containerID="0dad872019c123a1a70e1164abc0a863df30aca37ecc3c24a22b9fdd2a791d73" exitCode=0 Jul 21 18:28:04.608259 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:04.607914 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b" event={"ID":"d994b2fb-347a-4246-96b4-bcbb8c24f92b","Type":"ContainerDied","Data":"0dad872019c123a1a70e1164abc0a863df30aca37ecc3c24a22b9fdd2a791d73"} Jul 21 18:28:04.608259 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:04.607980 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b" event={"ID":"d994b2fb-347a-4246-96b4-bcbb8c24f92b","Type":"ContainerStarted","Data":"9fcbc1e00e44a82b16e21b10f09dbc74140c5cd8c7b18af8bc13d820293d48bd"} Jul 21 18:28:05.753066 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:05.753037 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b" Jul 21 18:28:05.791505 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:05.791452 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d994b2fb-347a-4246-96b4-bcbb8c24f92b-maas-api-service-ca\") pod \"d994b2fb-347a-4246-96b4-bcbb8c24f92b\" (UID: \"d994b2fb-347a-4246-96b4-bcbb8c24f92b\") " Jul 21 18:28:05.791774 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:05.791750 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d994b2fb-347a-4246-96b4-bcbb8c24f92b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d994b2fb-347a-4246-96b4-bcbb8c24f92b" (UID: "d994b2fb-347a-4246-96b4-bcbb8c24f92b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:28:05.892989 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:05.892888 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d994b2fb-347a-4246-96b4-bcbb8c24f92b-maas-api-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:28:06.619128 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:06.619099 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b" Jul 21 18:28:06.619297 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:06.619095 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b" event={"ID":"d994b2fb-347a-4246-96b4-bcbb8c24f92b","Type":"ContainerDied","Data":"9fcbc1e00e44a82b16e21b10f09dbc74140c5cd8c7b18af8bc13d820293d48bd"} Jul 21 18:28:06.619297 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:06.619228 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9fcbc1e00e44a82b16e21b10f09dbc74140c5cd8c7b18af8bc13d820293d48bd" Jul 21 18:28:13.649001 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:13.648956 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:16.231903 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:16.231870 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:28:18.261522 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:18.261482 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b"] Jul 21 18:28:18.262240 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:18.262216 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:18.263715 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:18.263689 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-24f6ad1b-rjs5b"] Jul 21 18:28:18.439249 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:18.439214 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:18.689826 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:18.689791 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:18.932264 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:18.932233 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:19.030588 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:19.030502 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:19.158805 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:19.158767 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:19.218219 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:19.218190 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d994b2fb-347a-4246-96b4-bcbb8c24f92b" path="/var/lib/kubelet/pods/d994b2fb-347a-4246-96b4-bcbb8c24f92b/volumes" Jul 21 18:28:19.476383 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:19.476347 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:20.010790 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:20.010757 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:21.998360 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:21.998285 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:22.254863 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:22.254776 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:22.428480 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:22.428446 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:22.728332 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:22.728301 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:22.828731 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:22.828698 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:28.634114 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:28.634072 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:28:38.489718 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:38.489678 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:28:47.035764 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.035726 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc"] Jul 21 18:28:47.036138 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.036051 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d994b2fb-347a-4246-96b4-bcbb8c24f92b" containerName="revoke-keys" Jul 21 18:28:47.036138 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.036063 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d994b2fb-347a-4246-96b4-bcbb8c24f92b" containerName="revoke-keys" Jul 21 18:28:47.036138 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.036135 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d994b2fb-347a-4246-96b4-bcbb8c24f92b" containerName="revoke-keys" Jul 21 18:28:47.039135 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.039113 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc" Jul 21 18:28:47.042268 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.042248 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:28:47.042664 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.042648 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 18:28:47.050234 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.050205 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc"] Jul 21 18:28:47.131518 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.131468 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63c260fe-4cc5-48f0-8397-cdf64289a39d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc\" (UID: \"63c260fe-4cc5-48f0-8397-cdf64289a39d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc" Jul 21 18:28:47.232473 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.232441 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63c260fe-4cc5-48f0-8397-cdf64289a39d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc\" (UID: \"63c260fe-4cc5-48f0-8397-cdf64289a39d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc" Jul 21 18:28:47.233087 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.233068 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63c260fe-4cc5-48f0-8397-cdf64289a39d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc\" (UID: \"63c260fe-4cc5-48f0-8397-cdf64289a39d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc" Jul 21 18:28:47.348774 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.348739 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc" Jul 21 18:28:47.476784 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.476760 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc"] Jul 21 18:28:47.478899 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:28:47.478865 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63c260fe_4cc5_48f0_8397_cdf64289a39d.slice/crio-509e9e0791e1d3b88317899a3ebb913afdeff3fb08fa1205d07ef223bb24c336 WatchSource:0}: Error finding container 509e9e0791e1d3b88317899a3ebb913afdeff3fb08fa1205d07ef223bb24c336: Status 404 returned error can't find the container with id 509e9e0791e1d3b88317899a3ebb913afdeff3fb08fa1205d07ef223bb24c336 Jul 21 18:28:47.832774 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.832740 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc" event={"ID":"63c260fe-4cc5-48f0-8397-cdf64289a39d","Type":"ContainerStarted","Data":"79e7e76f00fbfa5e9445dec791f33bc73de3963d268b034f04f23c500a8c1933"} Jul 21 18:28:47.832990 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.832790 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc" event={"ID":"63c260fe-4cc5-48f0-8397-cdf64289a39d","Type":"ContainerStarted","Data":"509e9e0791e1d3b88317899a3ebb913afdeff3fb08fa1205d07ef223bb24c336"} Jul 21 18:28:47.848786 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:47.848739 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc" podStartSLOduration=0.848723282 podStartE2EDuration="848.723282ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:28:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:28:47.847371524 +0000 UTC m=+1729.115682965" watchObservedRunningTime="2026-07-21 18:28:47.848723282 +0000 UTC m=+1729.117034722" Jul 21 18:28:48.838905 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:48.838865 2568 generic.go:356] "Generic (PLEG): container finished" podID="63c260fe-4cc5-48f0-8397-cdf64289a39d" containerID="79e7e76f00fbfa5e9445dec791f33bc73de3963d268b034f04f23c500a8c1933" exitCode=0 Jul 21 18:28:48.839299 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:48.838924 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc" event={"ID":"63c260fe-4cc5-48f0-8397-cdf64289a39d","Type":"ContainerDied","Data":"79e7e76f00fbfa5e9445dec791f33bc73de3963d268b034f04f23c500a8c1933"} Jul 21 18:28:49.970126 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:49.970099 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc" Jul 21 18:28:50.055911 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:50.055872 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63c260fe-4cc5-48f0-8397-cdf64289a39d-maas-api-service-ca\") pod \"63c260fe-4cc5-48f0-8397-cdf64289a39d\" (UID: \"63c260fe-4cc5-48f0-8397-cdf64289a39d\") " Jul 21 18:28:50.056234 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:50.056209 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63c260fe-4cc5-48f0-8397-cdf64289a39d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "63c260fe-4cc5-48f0-8397-cdf64289a39d" (UID: "63c260fe-4cc5-48f0-8397-cdf64289a39d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:28:50.156997 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:50.156912 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63c260fe-4cc5-48f0-8397-cdf64289a39d-maas-api-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:28:50.850097 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:50.850066 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc" Jul 21 18:28:50.850297 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:50.850063 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc" event={"ID":"63c260fe-4cc5-48f0-8397-cdf64289a39d","Type":"ContainerDied","Data":"509e9e0791e1d3b88317899a3ebb913afdeff3fb08fa1205d07ef223bb24c336"} Jul 21 18:28:50.850297 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:50.850208 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="509e9e0791e1d3b88317899a3ebb913afdeff3fb08fa1205d07ef223bb24c336" Jul 21 18:28:57.593096 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:28:57.593066 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:00.526305 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:00.526271 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:29:02.118128 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:02.118092 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc"] Jul 21 18:29:02.122862 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:02.122831 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ee15a55f-tj2fc"] Jul 21 18:29:02.189520 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:02.189491 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:02.269436 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:02.269406 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:02.505694 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:02.505614 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:02.735622 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:02.735590 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:03.056081 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:03.056044 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:03.218530 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:03.218494 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63c260fe-4cc5-48f0-8397-cdf64289a39d" path="/var/lib/kubelet/pods/63c260fe-4cc5-48f0-8397-cdf64289a39d/volumes" Jul 21 18:29:03.299584 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:03.299533 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:05.090780 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:05.090742 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:05.192593 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:05.192564 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:05.303826 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:05.303792 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:05.490393 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:05.490364 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:05.866110 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:05.866078 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:05.968284 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:05.968249 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:11.931866 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:11.931825 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:29:30.891821 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:30.891781 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f"] Jul 21 18:29:30.892242 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:30.892126 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63c260fe-4cc5-48f0-8397-cdf64289a39d" containerName="revoke-keys" Jul 21 18:29:30.892242 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:30.892137 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="63c260fe-4cc5-48f0-8397-cdf64289a39d" containerName="revoke-keys" Jul 21 18:29:30.892242 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:30.892207 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="63c260fe-4cc5-48f0-8397-cdf64289a39d" containerName="revoke-keys" Jul 21 18:29:30.895324 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:30.895308 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f" Jul 21 18:29:30.898164 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:30.898139 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:29:30.898301 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:30.898190 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 18:29:30.902658 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:30.902622 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f"] Jul 21 18:29:31.015485 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:31.015443 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a26438c1-fec5-405c-be8c-984ccece3605-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f\" (UID: \"a26438c1-fec5-405c-be8c-984ccece3605\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f" Jul 21 18:29:31.116301 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:31.116263 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a26438c1-fec5-405c-be8c-984ccece3605-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f\" (UID: \"a26438c1-fec5-405c-be8c-984ccece3605\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f" Jul 21 18:29:31.116982 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:31.116964 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a26438c1-fec5-405c-be8c-984ccece3605-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f\" (UID: \"a26438c1-fec5-405c-be8c-984ccece3605\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f" Jul 21 18:29:31.207537 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:31.207446 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f" Jul 21 18:29:31.333240 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:31.333215 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f"] Jul 21 18:29:31.335551 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:29:31.335523 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda26438c1_fec5_405c_be8c_984ccece3605.slice/crio-b62cae1b1376bca65f207bab2bbd5866e659aea0da41039ee8104dfd1f0dce95 WatchSource:0}: Error finding container b62cae1b1376bca65f207bab2bbd5866e659aea0da41039ee8104dfd1f0dce95: Status 404 returned error can't find the container with id b62cae1b1376bca65f207bab2bbd5866e659aea0da41039ee8104dfd1f0dce95 Jul 21 18:29:31.859814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:31.859787 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:32.061586 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:32.061555 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f" event={"ID":"a26438c1-fec5-405c-be8c-984ccece3605","Type":"ContainerStarted","Data":"db5da826d417617283e189d27b742f99f6af88ee3774a7bd9d57bfb02a2f4287"} Jul 21 18:29:32.062019 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:32.061614 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f" event={"ID":"a26438c1-fec5-405c-be8c-984ccece3605","Type":"ContainerStarted","Data":"b62cae1b1376bca65f207bab2bbd5866e659aea0da41039ee8104dfd1f0dce95"} Jul 21 18:29:32.078340 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:32.078292 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f" podStartSLOduration=2.078276596 podStartE2EDuration="2.078276596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:29:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:29:32.076372411 +0000 UTC m=+1773.344683852" watchObservedRunningTime="2026-07-21 18:29:32.078276596 +0000 UTC m=+1773.346588091" Jul 21 18:29:33.068328 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:33.068288 2568 generic.go:356] "Generic (PLEG): container finished" podID="a26438c1-fec5-405c-be8c-984ccece3605" containerID="db5da826d417617283e189d27b742f99f6af88ee3774a7bd9d57bfb02a2f4287" exitCode=0 Jul 21 18:29:33.068697 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:33.068375 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f" event={"ID":"a26438c1-fec5-405c-be8c-984ccece3605","Type":"ContainerDied","Data":"db5da826d417617283e189d27b742f99f6af88ee3774a7bd9d57bfb02a2f4287"} Jul 21 18:29:34.200971 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:34.200929 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f" Jul 21 18:29:34.344716 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:34.344687 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a26438c1-fec5-405c-be8c-984ccece3605-maas-api-service-ca\") pod \"a26438c1-fec5-405c-be8c-984ccece3605\" (UID: \"a26438c1-fec5-405c-be8c-984ccece3605\") " Jul 21 18:29:34.345125 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:34.345092 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a26438c1-fec5-405c-be8c-984ccece3605-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a26438c1-fec5-405c-be8c-984ccece3605" (UID: "a26438c1-fec5-405c-be8c-984ccece3605"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:29:34.446114 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:34.446078 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a26438c1-fec5-405c-be8c-984ccece3605-maas-api-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:29:35.079776 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:35.079740 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f" Jul 21 18:29:35.079776 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:35.079770 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f" event={"ID":"a26438c1-fec5-405c-be8c-984ccece3605","Type":"ContainerDied","Data":"b62cae1b1376bca65f207bab2bbd5866e659aea0da41039ee8104dfd1f0dce95"} Jul 21 18:29:35.080013 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:35.079813 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b62cae1b1376bca65f207bab2bbd5866e659aea0da41039ee8104dfd1f0dce95" Jul 21 18:29:41.329887 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:41.329855 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:44.130054 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:44.130019 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:29:45.976905 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:45.976870 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f"] Jul 21 18:29:45.982236 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:45.982206 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b6157a9f-lrq8f"] Jul 21 18:29:46.012186 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:46.012153 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:47.203640 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:47.203594 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:47.218609 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:47.218568 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a26438c1-fec5-405c-be8c-984ccece3605" path="/var/lib/kubelet/pods/a26438c1-fec5-405c-be8c-984ccece3605/volumes" Jul 21 18:29:48.324747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:48.324714 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:49.253177 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:49.253134 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:50.336390 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:50.336359 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:50.449947 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:50.449915 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:51.430253 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:51.430163 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:51.530530 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:51.530487 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:56.628560 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:56.628519 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:29:59.276247 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:59.276218 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:29:59.279598 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:59.279576 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:29:59.282769 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:59.282748 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:29:59.285506 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:59.285489 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:29:59.759447 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:59.759416 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:29:59.860764 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:29:59.860729 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:00.130389 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:00.130353 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744310-btlwd"] Jul 21 18:30:00.130760 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:00.130738 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a26438c1-fec5-405c-be8c-984ccece3605" containerName="revoke-keys" Jul 21 18:30:00.131161 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:00.130764 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a26438c1-fec5-405c-be8c-984ccece3605" containerName="revoke-keys" Jul 21 18:30:00.131161 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:00.130885 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a26438c1-fec5-405c-be8c-984ccece3605" containerName="revoke-keys" Jul 21 18:30:00.134304 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:00.134284 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7f620e25-29744310-zxpbg"] Jul 21 18:30:00.134421 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:00.134406 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744310-btlwd" Jul 21 18:30:00.136942 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:00.136912 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:30:00.138334 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:00.138314 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7f620e25-29744310-zxpbg" Jul 21 18:30:00.141044 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:00.141007 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744310-btlwd"] Jul 21 18:30:00.146624 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:00.146605 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744310-btlwd" Jul 21 18:30:00.154501 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:00.154419 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7f620e25-29744310-zxpbg" Jul 21 18:30:00.155024 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:00.155004 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7f620e25-29744310-zxpbg"] Jul 21 18:30:00.326761 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:00.326735 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7f620e25-29744310-zxpbg"] Jul 21 18:30:00.328192 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:30:00.328155 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod728cacd2_2301_49fa_9a1a_c4d1c3204e6b.slice/crio-43f455baf2abd690bd08a0f9cb5bdda8f7138265c5771b3e883b0a026fd1278c WatchSource:0}: Error finding container 43f455baf2abd690bd08a0f9cb5bdda8f7138265c5771b3e883b0a026fd1278c: Status 404 returned error can't find the container with id 43f455baf2abd690bd08a0f9cb5bdda8f7138265c5771b3e883b0a026fd1278c Jul 21 18:30:00.500958 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:00.500914 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744310-btlwd"] Jul 21 18:30:00.503201 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:30:00.503168 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1aa9a8b_932f_4bc6_a9c0_abdec2c2e440.slice/crio-6f480efd0a21df824df1a549d32f03824ce0907a229b80572a9cab1c9ca379f2 WatchSource:0}: Error finding container 6f480efd0a21df824df1a549d32f03824ce0907a229b80572a9cab1c9ca379f2: Status 404 returned error can't find the container with id 6f480efd0a21df824df1a549d32f03824ce0907a229b80572a9cab1c9ca379f2 Jul 21 18:30:01.229109 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:01.229063 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7f620e25-29744310-zxpbg" event={"ID":"728cacd2-2301-49fa-9a1a-c4d1c3204e6b","Type":"ContainerStarted","Data":"3faaf0fe2f255cd186e4c412c36dc2ee8e77756a713298374537ecaebeb35808"} Jul 21 18:30:01.229350 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:01.229163 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7f620e25-29744310-zxpbg" event={"ID":"728cacd2-2301-49fa-9a1a-c4d1c3204e6b","Type":"ContainerStarted","Data":"43f455baf2abd690bd08a0f9cb5bdda8f7138265c5771b3e883b0a026fd1278c"} Jul 21 18:30:01.230431 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:01.230402 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744310-btlwd" event={"ID":"a1aa9a8b-932f-4bc6-a9c0-abdec2c2e440","Type":"ContainerStarted","Data":"391b56fb99fe7e91753e73de9d88abf0c7928835b290432c9bcfd3ed2c7937c4"} Jul 21 18:30:01.230548 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:01.230440 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744310-btlwd" event={"ID":"a1aa9a8b-932f-4bc6-a9c0-abdec2c2e440","Type":"ContainerStarted","Data":"6f480efd0a21df824df1a549d32f03824ce0907a229b80572a9cab1c9ca379f2"} Jul 21 18:30:01.261967 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:01.261897 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7f620e25-29744310-zxpbg" podStartSLOduration=1.261882597 podStartE2EDuration="1.261882597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:30:01.256886646 +0000 UTC m=+1802.525198082" watchObservedRunningTime="2026-07-21 18:30:01.261882597 +0000 UTC m=+1802.530194038" Jul 21 18:30:01.290311 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:01.290256 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744310-btlwd" podStartSLOduration=1.290236788 podStartE2EDuration="1.290236788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:30:01.289524526 +0000 UTC m=+1802.557835978" watchObservedRunningTime="2026-07-21 18:30:01.290236788 +0000 UTC m=+1802.558548232" Jul 21 18:30:02.237267 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:02.237234 2568 generic.go:356] "Generic (PLEG): container finished" podID="728cacd2-2301-49fa-9a1a-c4d1c3204e6b" containerID="3faaf0fe2f255cd186e4c412c36dc2ee8e77756a713298374537ecaebeb35808" exitCode=0 Jul 21 18:30:02.237678 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:02.237331 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7f620e25-29744310-zxpbg" event={"ID":"728cacd2-2301-49fa-9a1a-c4d1c3204e6b","Type":"ContainerDied","Data":"3faaf0fe2f255cd186e4c412c36dc2ee8e77756a713298374537ecaebeb35808"} Jul 21 18:30:02.238753 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:02.238730 2568 generic.go:356] "Generic (PLEG): container finished" podID="a1aa9a8b-932f-4bc6-a9c0-abdec2c2e440" containerID="391b56fb99fe7e91753e73de9d88abf0c7928835b290432c9bcfd3ed2c7937c4" exitCode=0 Jul 21 18:30:02.238854 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:02.238790 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744310-btlwd" event={"ID":"a1aa9a8b-932f-4bc6-a9c0-abdec2c2e440","Type":"ContainerDied","Data":"391b56fb99fe7e91753e73de9d88abf0c7928835b290432c9bcfd3ed2c7937c4"} Jul 21 18:30:03.374798 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:03.374773 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744310-btlwd" Jul 21 18:30:03.407335 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:03.407298 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7f620e25-29744310-zxpbg" Jul 21 18:30:04.250851 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:04.250817 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7f620e25-29744310-zxpbg" Jul 21 18:30:04.251059 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:04.250859 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7f620e25-29744310-zxpbg" event={"ID":"728cacd2-2301-49fa-9a1a-c4d1c3204e6b","Type":"ContainerDied","Data":"43f455baf2abd690bd08a0f9cb5bdda8f7138265c5771b3e883b0a026fd1278c"} Jul 21 18:30:04.251059 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:04.250906 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43f455baf2abd690bd08a0f9cb5bdda8f7138265c5771b3e883b0a026fd1278c" Jul 21 18:30:04.252153 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:04.252133 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744310-btlwd" event={"ID":"a1aa9a8b-932f-4bc6-a9c0-abdec2c2e440","Type":"ContainerDied","Data":"6f480efd0a21df824df1a549d32f03824ce0907a229b80572a9cab1c9ca379f2"} Jul 21 18:30:04.252247 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:04.252158 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744310-btlwd" Jul 21 18:30:04.252247 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:04.252166 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f480efd0a21df824df1a549d32f03824ce0907a229b80572a9cab1c9ca379f2" Jul 21 18:30:13.558930 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:13.558898 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:13.618233 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:13.618194 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:15.179188 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.179154 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g"] Jul 21 18:30:15.179922 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.179757 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1aa9a8b-932f-4bc6-a9c0-abdec2c2e440" containerName="cleanup" Jul 21 18:30:15.179922 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.179779 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1aa9a8b-932f-4bc6-a9c0-abdec2c2e440" containerName="cleanup" Jul 21 18:30:15.179922 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.179794 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="728cacd2-2301-49fa-9a1a-c4d1c3204e6b" containerName="cleanup" Jul 21 18:30:15.179922 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.179802 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="728cacd2-2301-49fa-9a1a-c4d1c3204e6b" containerName="cleanup" Jul 21 18:30:15.180185 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.179930 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1aa9a8b-932f-4bc6-a9c0-abdec2c2e440" containerName="cleanup" Jul 21 18:30:15.180185 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.179963 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="728cacd2-2301-49fa-9a1a-c4d1c3204e6b" containerName="cleanup" Jul 21 18:30:15.183074 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.183053 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g" Jul 21 18:30:15.185690 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.185672 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:30:15.185783 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.185739 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 18:30:15.191584 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.191561 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g"] Jul 21 18:30:15.306634 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.306595 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbcd8cff-1666-4859-ab2e-4e21e889192d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g\" (UID: \"bbcd8cff-1666-4859-ab2e-4e21e889192d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g" Jul 21 18:30:15.407130 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.407093 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbcd8cff-1666-4859-ab2e-4e21e889192d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g\" (UID: \"bbcd8cff-1666-4859-ab2e-4e21e889192d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g" Jul 21 18:30:15.407723 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.407701 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbcd8cff-1666-4859-ab2e-4e21e889192d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g\" (UID: \"bbcd8cff-1666-4859-ab2e-4e21e889192d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g" Jul 21 18:30:15.494258 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.494177 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g" Jul 21 18:30:15.626485 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:15.626456 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g"] Jul 21 18:30:15.628305 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:30:15.628278 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbcd8cff_1666_4859_ab2e_4e21e889192d.slice/crio-44e324ee7cffe82131f595bcf35da5674cd511c94ffc0888ea90f94d1d81741e WatchSource:0}: Error finding container 44e324ee7cffe82131f595bcf35da5674cd511c94ffc0888ea90f94d1d81741e: Status 404 returned error can't find the container with id 44e324ee7cffe82131f595bcf35da5674cd511c94ffc0888ea90f94d1d81741e Jul 21 18:30:16.317717 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:16.317684 2568 generic.go:356] "Generic (PLEG): container finished" podID="bbcd8cff-1666-4859-ab2e-4e21e889192d" containerID="2c7b61c8223a75fb0bbb5b30b54b7c7e551d0849cdc68a1f1ad9e4e1f08b1b1c" exitCode=0 Jul 21 18:30:16.318146 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:16.317768 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g" event={"ID":"bbcd8cff-1666-4859-ab2e-4e21e889192d","Type":"ContainerDied","Data":"2c7b61c8223a75fb0bbb5b30b54b7c7e551d0849cdc68a1f1ad9e4e1f08b1b1c"} Jul 21 18:30:16.318146 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:16.317823 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g" event={"ID":"bbcd8cff-1666-4859-ab2e-4e21e889192d","Type":"ContainerStarted","Data":"44e324ee7cffe82131f595bcf35da5674cd511c94ffc0888ea90f94d1d81741e"} Jul 21 18:30:17.451975 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:17.451950 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g" Jul 21 18:30:17.627533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:17.627498 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbcd8cff-1666-4859-ab2e-4e21e889192d-maas-api-service-ca\") pod \"bbcd8cff-1666-4859-ab2e-4e21e889192d\" (UID: \"bbcd8cff-1666-4859-ab2e-4e21e889192d\") " Jul 21 18:30:17.627891 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:17.627868 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbcd8cff-1666-4859-ab2e-4e21e889192d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bbcd8cff-1666-4859-ab2e-4e21e889192d" (UID: "bbcd8cff-1666-4859-ab2e-4e21e889192d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:30:17.728785 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:17.728747 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbcd8cff-1666-4859-ab2e-4e21e889192d-maas-api-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:30:18.329750 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:18.329723 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g" Jul 21 18:30:18.329897 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:18.329722 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g" event={"ID":"bbcd8cff-1666-4859-ab2e-4e21e889192d","Type":"ContainerDied","Data":"44e324ee7cffe82131f595bcf35da5674cd511c94ffc0888ea90f94d1d81741e"} Jul 21 18:30:18.329897 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:18.329839 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44e324ee7cffe82131f595bcf35da5674cd511c94ffc0888ea90f94d1d81741e" Jul 21 18:30:25.629880 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:25.629833 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:25.636951 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:25.636912 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7f620e25-29744310-zxpbg"] Jul 21 18:30:25.639195 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:25.639175 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-7f620e25-29744310-zxpbg"] Jul 21 18:30:27.219153 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:27.219117 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="728cacd2-2301-49fa-9a1a-c4d1c3204e6b" path="/var/lib/kubelet/pods/728cacd2-2301-49fa-9a1a-c4d1c3204e6b/volumes" Jul 21 18:30:28.627912 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:28.627873 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:30:30.257250 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:30.257216 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g"] Jul 21 18:30:30.259203 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:30.259164 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:30.259569 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:30.259548 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7f620e25-6ct9g"] Jul 21 18:30:30.413116 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:30.413075 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:30.695012 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:30.694978 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:31.179847 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:31.179817 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:31.218303 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:31.218273 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbcd8cff-1666-4859-ab2e-4e21e889192d" path="/var/lib/kubelet/pods/bbcd8cff-1666-4859-ab2e-4e21e889192d/volumes" Jul 21 18:30:31.267167 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:31.267135 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:31.366816 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:31.366785 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:32.818215 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:32.818170 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:39.415245 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:39.415214 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:39.524902 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:39.524858 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:39.620972 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:39.620894 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:39.715899 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:39.715798 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:30:40.109144 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:30:40.109099 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:06.091440 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:06.091405 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:13.062846 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:13.062801 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:13.135072 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:13.135039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:13.244972 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:13.244918 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:29.177906 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:29.177814 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:35.545966 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:35.545895 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:35.612050 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:35.612007 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:35.693800 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:35.693757 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:35.796410 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:35.796320 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:46.290578 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:46.290545 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:47.092737 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:47.092702 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:55.074269 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:55.074231 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:55.147950 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:55.147905 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:55.262189 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:55.262154 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:55.359076 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:55.359044 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:56.154090 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:56.154058 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:56.250390 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:56.250352 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:56.349881 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:56.349845 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:31:57.133067 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:57.133030 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:31:58.006718 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:31:58.006684 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:04.821148 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:04.821109 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:04.921913 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:04.921866 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:05.237312 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:05.237277 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:06.837440 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:06.837399 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:32:11.572282 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:11.572246 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:11.643283 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:11.643252 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:11.802860 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:11.802822 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:12.095887 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:12.095856 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:12.315088 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:12.315039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:14.828733 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:14.828695 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:14.919270 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:14.919230 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z"] Jul 21 18:32:14.919630 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:14.919616 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbcd8cff-1666-4859-ab2e-4e21e889192d" containerName="revoke-keys" Jul 21 18:32:14.919674 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:14.919632 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbcd8cff-1666-4859-ab2e-4e21e889192d" containerName="revoke-keys" Jul 21 18:32:14.919707 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:14.919691 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbcd8cff-1666-4859-ab2e-4e21e889192d" containerName="revoke-keys" Jul 21 18:32:14.922880 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:14.922858 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:14.925618 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:14.925595 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 18:32:14.925762 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:14.925595 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 18:32:14.926571 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:14.926548 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-26807d71-serving-cert\"" Jul 21 18:32:14.926676 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:14.926551 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sh2jm\"" Jul 21 18:32:14.931696 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:14.931636 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:14.934500 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:14.934474 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z"] Jul 21 18:32:15.012119 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.012088 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:15.023318 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.023284 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-maas-api-tls\") pod \"maas-api-e2e-mt-26807d71-767db55888-bp58z\" (UID: \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:15.023478 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.023329 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-openshift-service-ca\") pod \"maas-api-e2e-mt-26807d71-767db55888-bp58z\" (UID: \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:15.023478 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.023403 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftcss\" (UniqueName: \"kubernetes.io/projected/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-kube-api-access-ftcss\") pod \"maas-api-e2e-mt-26807d71-767db55888-bp58z\" (UID: \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:15.124566 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.124527 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-maas-api-tls\") pod \"maas-api-e2e-mt-26807d71-767db55888-bp58z\" (UID: \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:15.124756 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.124572 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-openshift-service-ca\") pod \"maas-api-e2e-mt-26807d71-767db55888-bp58z\" (UID: \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:15.124756 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.124615 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftcss\" (UniqueName: \"kubernetes.io/projected/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-kube-api-access-ftcss\") pod \"maas-api-e2e-mt-26807d71-767db55888-bp58z\" (UID: \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:15.124756 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:32:15.124681 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-serving-cert: secret "maas-api-e2e-mt-26807d71-serving-cert" not found Jul 21 18:32:15.124886 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:32:15.124760 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-maas-api-tls podName:67502e54-bddb-47a2-8f6f-fa4bd347bf2e nodeName:}" failed. No retries permitted until 2026-07-21 18:32:15.624742604 +0000 UTC m=+1936.893054024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-maas-api-tls") pod "maas-api-e2e-mt-26807d71-767db55888-bp58z" (UID: "67502e54-bddb-47a2-8f6f-fa4bd347bf2e") : secret "maas-api-e2e-mt-26807d71-serving-cert" not found Jul 21 18:32:15.125309 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.125289 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-openshift-service-ca\") pod \"maas-api-e2e-mt-26807d71-767db55888-bp58z\" (UID: \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:15.136562 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.136529 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftcss\" (UniqueName: \"kubernetes.io/projected/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-kube-api-access-ftcss\") pod \"maas-api-e2e-mt-26807d71-767db55888-bp58z\" (UID: \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:15.272411 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.272351 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:15.578662 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.578545 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:15.630626 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.630571 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-maas-api-tls\") pod \"maas-api-e2e-mt-26807d71-767db55888-bp58z\" (UID: \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:15.633043 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.633014 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-maas-api-tls\") pod \"maas-api-e2e-mt-26807d71-767db55888-bp58z\" (UID: \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:15.678905 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.678862 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:15.834904 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.834810 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:15.980243 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.980218 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z"] Jul 21 18:32:15.982502 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:32:15.982474 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67502e54_bddb_47a2_8f6f_fa4bd347bf2e.slice/crio-10c291e71a9a432b71b8c6e7a0adf23bc311f3d5b31e718af87a55bb1fb22629 WatchSource:0}: Error finding container 10c291e71a9a432b71b8c6e7a0adf23bc311f3d5b31e718af87a55bb1fb22629: Status 404 returned error can't find the container with id 10c291e71a9a432b71b8c6e7a0adf23bc311f3d5b31e718af87a55bb1fb22629 Jul 21 18:32:15.983778 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:15.983761 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 18:32:16.962063 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:16.962019 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" event={"ID":"67502e54-bddb-47a2-8f6f-fa4bd347bf2e","Type":"ContainerStarted","Data":"10c291e71a9a432b71b8c6e7a0adf23bc311f3d5b31e718af87a55bb1fb22629"} Jul 21 18:32:17.833930 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:17.833890 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:32:18.976785 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:18.976742 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" event={"ID":"67502e54-bddb-47a2-8f6f-fa4bd347bf2e","Type":"ContainerStarted","Data":"d60e4b1c7f46aece2196b14dae9975ebbf69f4506cbb5d326a7668deb004b534"} Jul 21 18:32:18.977208 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:18.976816 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:18.993305 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:18.993249 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" podStartSLOduration=2.84229969 podStartE2EDuration="4.993230689s" totalImagesPullingTime="2.150930999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:32:18.993133687 +0000 UTC m=+1940.261445129" watchObservedRunningTime="2026-07-21 18:32:18.993230689 +0000 UTC m=+1940.261542131" Jul 21 18:32:19.320509 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:19.320419 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:20.031698 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:20.031666 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:20.125238 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:20.125208 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:20.441345 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:20.441308 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:24.987559 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:24.987532 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:26.674652 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:26.674608 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:26.806840 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:26.806809 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:27.056333 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:27.056251 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw"] Jul 21 18:32:27.059872 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:27.059854 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw" Jul 21 18:32:27.062304 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:27.062283 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:32:27.067534 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:27.067507 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw"] Jul 21 18:32:27.235144 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:27.235103 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/801a302d-695d-414d-93e7-2f82c596bd01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-26807d71-rt9lw\" (UID: \"801a302d-695d-414d-93e7-2f82c596bd01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw" Jul 21 18:32:27.336380 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:27.336287 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/801a302d-695d-414d-93e7-2f82c596bd01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-26807d71-rt9lw\" (UID: \"801a302d-695d-414d-93e7-2f82c596bd01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw" Jul 21 18:32:27.337051 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:27.337029 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/801a302d-695d-414d-93e7-2f82c596bd01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-26807d71-rt9lw\" (UID: \"801a302d-695d-414d-93e7-2f82c596bd01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw" Jul 21 18:32:27.372074 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:27.372035 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw" Jul 21 18:32:27.510728 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:27.510703 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw"] Jul 21 18:32:27.512482 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:32:27.512452 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod801a302d_695d_414d_93e7_2f82c596bd01.slice/crio-3b1876ac74464989d6db6fb9ce08a4e1c9c7d0146068eb13ea3216dbed77b829 WatchSource:0}: Error finding container 3b1876ac74464989d6db6fb9ce08a4e1c9c7d0146068eb13ea3216dbed77b829: Status 404 returned error can't find the container with id 3b1876ac74464989d6db6fb9ce08a4e1c9c7d0146068eb13ea3216dbed77b829 Jul 21 18:32:28.026986 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:28.026870 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw" event={"ID":"801a302d-695d-414d-93e7-2f82c596bd01","Type":"ContainerStarted","Data":"87e7a977f65e5929c71a745baa153a6b997a772ad6b2d29a329a040ae66e83b7"} Jul 21 18:32:28.026986 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:28.026957 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw" event={"ID":"801a302d-695d-414d-93e7-2f82c596bd01","Type":"ContainerStarted","Data":"3b1876ac74464989d6db6fb9ce08a4e1c9c7d0146068eb13ea3216dbed77b829"} Jul 21 18:32:28.043927 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:28.043869 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw" podStartSLOduration=1.043851101 podStartE2EDuration="1.043851101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:32:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:32:28.040922014 +0000 UTC m=+1949.309233455" watchObservedRunningTime="2026-07-21 18:32:28.043851101 +0000 UTC m=+1949.312162542" Jul 21 18:32:29.033702 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:29.033667 2568 generic.go:356] "Generic (PLEG): container finished" podID="801a302d-695d-414d-93e7-2f82c596bd01" containerID="87e7a977f65e5929c71a745baa153a6b997a772ad6b2d29a329a040ae66e83b7" exitCode=0 Jul 21 18:32:29.034140 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:29.033742 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw" event={"ID":"801a302d-695d-414d-93e7-2f82c596bd01","Type":"ContainerDied","Data":"87e7a977f65e5929c71a745baa153a6b997a772ad6b2d29a329a040ae66e83b7"} Jul 21 18:32:30.172101 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:30.172070 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw" Jul 21 18:32:30.260692 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:30.260651 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/801a302d-695d-414d-93e7-2f82c596bd01-maas-api-service-ca\") pod \"801a302d-695d-414d-93e7-2f82c596bd01\" (UID: \"801a302d-695d-414d-93e7-2f82c596bd01\") " Jul 21 18:32:30.261032 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:30.261008 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/801a302d-695d-414d-93e7-2f82c596bd01-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "801a302d-695d-414d-93e7-2f82c596bd01" (UID: "801a302d-695d-414d-93e7-2f82c596bd01"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:32:30.362058 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:30.362028 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/801a302d-695d-414d-93e7-2f82c596bd01-maas-api-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:32:31.045908 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:31.045878 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw" Jul 21 18:32:31.045908 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:31.045902 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw" event={"ID":"801a302d-695d-414d-93e7-2f82c596bd01","Type":"ContainerDied","Data":"3b1876ac74464989d6db6fb9ce08a4e1c9c7d0146068eb13ea3216dbed77b829"} Jul 21 18:32:31.046152 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:31.045968 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b1876ac74464989d6db6fb9ce08a4e1c9c7d0146068eb13ea3216dbed77b829" Jul 21 18:32:37.721802 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:37.721765 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z"] Jul 21 18:32:37.722370 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:37.722002 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" podUID="67502e54-bddb-47a2-8f6f-fa4bd347bf2e" containerName="maas-api" containerID="cri-o://d60e4b1c7f46aece2196b14dae9975ebbf69f4506cbb5d326a7668deb004b534" gracePeriod=30 Jul 21 18:32:37.751576 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:37.751544 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:37.971798 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:37.971773 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:38.014163 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.014053 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-openshift-service-ca\") pod \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\" (UID: \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\") " Jul 21 18:32:38.014331 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.014185 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-maas-api-tls\") pod \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\" (UID: \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\") " Jul 21 18:32:38.014331 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.014289 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftcss\" (UniqueName: \"kubernetes.io/projected/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-kube-api-access-ftcss\") pod \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\" (UID: \"67502e54-bddb-47a2-8f6f-fa4bd347bf2e\") " Jul 21 18:32:38.014498 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.014473 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "67502e54-bddb-47a2-8f6f-fa4bd347bf2e" (UID: "67502e54-bddb-47a2-8f6f-fa4bd347bf2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:32:38.014562 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.014542 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-openshift-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:32:38.016405 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.016378 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "67502e54-bddb-47a2-8f6f-fa4bd347bf2e" (UID: "67502e54-bddb-47a2-8f6f-fa4bd347bf2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:32:38.016514 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.016388 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-kube-api-access-ftcss" (OuterVolumeSpecName: "kube-api-access-ftcss") pod "67502e54-bddb-47a2-8f6f-fa4bd347bf2e" (UID: "67502e54-bddb-47a2-8f6f-fa4bd347bf2e"). InnerVolumeSpecName "kube-api-access-ftcss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:32:38.084293 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.084254 2568 generic.go:356] "Generic (PLEG): container finished" podID="67502e54-bddb-47a2-8f6f-fa4bd347bf2e" containerID="d60e4b1c7f46aece2196b14dae9975ebbf69f4506cbb5d326a7668deb004b534" exitCode=0 Jul 21 18:32:38.084445 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.084303 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" Jul 21 18:32:38.084445 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.084345 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" event={"ID":"67502e54-bddb-47a2-8f6f-fa4bd347bf2e","Type":"ContainerDied","Data":"d60e4b1c7f46aece2196b14dae9975ebbf69f4506cbb5d326a7668deb004b534"} Jul 21 18:32:38.084445 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.084397 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z" event={"ID":"67502e54-bddb-47a2-8f6f-fa4bd347bf2e","Type":"ContainerDied","Data":"10c291e71a9a432b71b8c6e7a0adf23bc311f3d5b31e718af87a55bb1fb22629"} Jul 21 18:32:38.084445 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.084425 2568 scope.go:117] "RemoveContainer" containerID="d60e4b1c7f46aece2196b14dae9975ebbf69f4506cbb5d326a7668deb004b534" Jul 21 18:32:38.094259 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.094239 2568 scope.go:117] "RemoveContainer" containerID="d60e4b1c7f46aece2196b14dae9975ebbf69f4506cbb5d326a7668deb004b534" Jul 21 18:32:38.094521 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:32:38.094501 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d60e4b1c7f46aece2196b14dae9975ebbf69f4506cbb5d326a7668deb004b534\": container with ID starting with d60e4b1c7f46aece2196b14dae9975ebbf69f4506cbb5d326a7668deb004b534 not found: ID does not exist" containerID="d60e4b1c7f46aece2196b14dae9975ebbf69f4506cbb5d326a7668deb004b534" Jul 21 18:32:38.094570 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.094531 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d60e4b1c7f46aece2196b14dae9975ebbf69f4506cbb5d326a7668deb004b534"} err="failed to get container status \"d60e4b1c7f46aece2196b14dae9975ebbf69f4506cbb5d326a7668deb004b534\": rpc error: code = NotFound desc = could not find container \"d60e4b1c7f46aece2196b14dae9975ebbf69f4506cbb5d326a7668deb004b534\": container with ID starting with d60e4b1c7f46aece2196b14dae9975ebbf69f4506cbb5d326a7668deb004b534 not found: ID does not exist" Jul 21 18:32:38.106747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.106716 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z"] Jul 21 18:32:38.111021 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.110990 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-26807d71-767db55888-bp58z"] Jul 21 18:32:38.114901 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.114875 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftcss\" (UniqueName: \"kubernetes.io/projected/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-kube-api-access-ftcss\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:32:38.114901 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:38.114901 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67502e54-bddb-47a2-8f6f-fa4bd347bf2e-maas-api-tls\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:32:39.218727 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:39.218689 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67502e54-bddb-47a2-8f6f-fa4bd347bf2e" path="/var/lib/kubelet/pods/67502e54-bddb-47a2-8f6f-fa4bd347bf2e/volumes" Jul 21 18:32:40.725839 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:40.725803 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:32:42.135295 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:42.135250 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:42.136771 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:42.136736 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw"] Jul 21 18:32:42.138863 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:42.138838 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-26807d71-rt9lw"] Jul 21 18:32:42.508496 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:42.508402 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:43.218410 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:43.218375 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="801a302d-695d-414d-93e7-2f82c596bd01" path="/var/lib/kubelet/pods/801a302d-695d-414d-93e7-2f82c596bd01/volumes" Jul 21 18:32:48.286126 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:48.286094 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:48.354373 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:48.354341 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:48.556237 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:48.556160 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:48.812604 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:48.812523 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:48.929970 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:48.927404 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:49.025672 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:49.025636 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:49.140040 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:49.140005 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:55.237548 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:55.237473 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:32:56.755226 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:32:56.755188 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:01.679709 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:01.679675 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:01.780130 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:01.780099 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:08.476216 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:08.476178 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:08.879178 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:08.879140 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:08.983248 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:08.983212 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:11.872025 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.871991 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:11.940691 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.940654 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz"] Jul 21 18:33:11.941056 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.941041 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="801a302d-695d-414d-93e7-2f82c596bd01" containerName="revoke-keys" Jul 21 18:33:11.941056 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.941056 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="801a302d-695d-414d-93e7-2f82c596bd01" containerName="revoke-keys" Jul 21 18:33:11.941171 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.941067 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67502e54-bddb-47a2-8f6f-fa4bd347bf2e" containerName="maas-api" Jul 21 18:33:11.941171 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.941073 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="67502e54-bddb-47a2-8f6f-fa4bd347bf2e" containerName="maas-api" Jul 21 18:33:11.941171 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.941163 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="801a302d-695d-414d-93e7-2f82c596bd01" containerName="revoke-keys" Jul 21 18:33:11.941274 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.941176 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="67502e54-bddb-47a2-8f6f-fa4bd347bf2e" containerName="maas-api" Jul 21 18:33:11.944314 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.944292 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:33:11.946745 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.946717 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-15fecbf7-serving-cert\"" Jul 21 18:33:11.946873 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.946746 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 18:33:11.946975 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.946957 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 18:33:11.947289 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.947267 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sh2jm\"" Jul 21 18:33:11.954191 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.954163 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz"] Jul 21 18:33:11.961281 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:11.961255 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:12.014326 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.014286 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6qfw\" (UniqueName: \"kubernetes.io/projected/c66188f9-6b68-44cc-994b-b7dce98883c2-kube-api-access-v6qfw\") pod \"maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz\" (UID: \"c66188f9-6b68-44cc-994b-b7dce98883c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:33:12.014499 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.014350 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c66188f9-6b68-44cc-994b-b7dce98883c2-openshift-service-ca\") pod \"maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz\" (UID: \"c66188f9-6b68-44cc-994b-b7dce98883c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:33:12.014499 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.014410 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c66188f9-6b68-44cc-994b-b7dce98883c2-maas-api-tls\") pod \"maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz\" (UID: \"c66188f9-6b68-44cc-994b-b7dce98883c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:33:12.029980 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.029911 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:12.115606 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.115564 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6qfw\" (UniqueName: \"kubernetes.io/projected/c66188f9-6b68-44cc-994b-b7dce98883c2-kube-api-access-v6qfw\") pod \"maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz\" (UID: \"c66188f9-6b68-44cc-994b-b7dce98883c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:33:12.115606 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.115613 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c66188f9-6b68-44cc-994b-b7dce98883c2-openshift-service-ca\") pod \"maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz\" (UID: \"c66188f9-6b68-44cc-994b-b7dce98883c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:33:12.115834 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.115652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c66188f9-6b68-44cc-994b-b7dce98883c2-maas-api-tls\") pod \"maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz\" (UID: \"c66188f9-6b68-44cc-994b-b7dce98883c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:33:12.115834 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:33:12.115799 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-serving-cert: secret "maas-api-e2e-mt-15fecbf7-serving-cert" not found Jul 21 18:33:12.115915 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:33:12.115867 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c66188f9-6b68-44cc-994b-b7dce98883c2-maas-api-tls podName:c66188f9-6b68-44cc-994b-b7dce98883c2 nodeName:}" failed. No retries permitted until 2026-07-21 18:33:12.615848975 +0000 UTC m=+1993.884160394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c66188f9-6b68-44cc-994b-b7dce98883c2-maas-api-tls") pod "maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" (UID: "c66188f9-6b68-44cc-994b-b7dce98883c2") : secret "maas-api-e2e-mt-15fecbf7-serving-cert" not found Jul 21 18:33:12.116449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.116428 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c66188f9-6b68-44cc-994b-b7dce98883c2-openshift-service-ca\") pod \"maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz\" (UID: \"c66188f9-6b68-44cc-994b-b7dce98883c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:33:12.127732 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.127661 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6qfw\" (UniqueName: \"kubernetes.io/projected/c66188f9-6b68-44cc-994b-b7dce98883c2-kube-api-access-v6qfw\") pod \"maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz\" (UID: \"c66188f9-6b68-44cc-994b-b7dce98883c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:33:12.446270 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.446184 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:12.543407 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.543372 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:12.621606 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.621569 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c66188f9-6b68-44cc-994b-b7dce98883c2-maas-api-tls\") pod \"maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz\" (UID: \"c66188f9-6b68-44cc-994b-b7dce98883c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:33:12.623991 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.623967 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c66188f9-6b68-44cc-994b-b7dce98883c2-maas-api-tls\") pod \"maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz\" (UID: \"c66188f9-6b68-44cc-994b-b7dce98883c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:33:12.644274 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.644241 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:12.855666 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.855629 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:33:12.991085 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:12.991048 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz"] Jul 21 18:33:12.992755 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:33:12.992728 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc66188f9_6b68_44cc_994b_b7dce98883c2.slice/crio-8ebbaf4596ae24fae7d06af6003834fe52c08d024e3cc37e5f7d5e589621043e WatchSource:0}: Error finding container 8ebbaf4596ae24fae7d06af6003834fe52c08d024e3cc37e5f7d5e589621043e: Status 404 returned error can't find the container with id 8ebbaf4596ae24fae7d06af6003834fe52c08d024e3cc37e5f7d5e589621043e Jul 21 18:33:13.283655 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:13.283571 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" event={"ID":"c66188f9-6b68-44cc-994b-b7dce98883c2","Type":"ContainerStarted","Data":"8ebbaf4596ae24fae7d06af6003834fe52c08d024e3cc37e5f7d5e589621043e"} Jul 21 18:33:14.290807 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:14.290762 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" event={"ID":"c66188f9-6b68-44cc-994b-b7dce98883c2","Type":"ContainerStarted","Data":"b504288e0b4a0720fd1d5e69d3462789ea735cdf7f12f3ecce6084c6cdccf34a"} Jul 21 18:33:14.291237 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:14.290915 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:33:14.308624 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:14.308574 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" podStartSLOduration=2.163105327 podStartE2EDuration="3.308559479s" totalImagesPullingTime="1.145454152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:33:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:33:14.307868459 +0000 UTC m=+1995.576179915" watchObservedRunningTime="2026-07-21 18:33:14.308559479 +0000 UTC m=+1995.576870898" Jul 21 18:33:14.648267 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:14.648232 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:16.366284 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:16.366250 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:18.228425 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:18.228388 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:33:20.302190 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:20.302164 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:33:33.656601 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:33.656568 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:33:36.832113 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:36.832074 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:33:51.740086 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:51.740052 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:33:54.912258 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:54.912223 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq"] Jul 21 18:33:54.916041 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:54.916018 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq" Jul 21 18:33:54.918714 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:54.918694 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:33:54.923971 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:54.923949 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq"] Jul 21 18:33:54.977551 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:54.977512 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c96c9640-958b-4387-87b3-ce636c47a372-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq\" (UID: \"c96c9640-958b-4387-87b3-ce636c47a372\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq" Jul 21 18:33:55.078838 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:55.078800 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c96c9640-958b-4387-87b3-ce636c47a372-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq\" (UID: \"c96c9640-958b-4387-87b3-ce636c47a372\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq" Jul 21 18:33:55.079526 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:55.079499 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c96c9640-958b-4387-87b3-ce636c47a372-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq\" (UID: \"c96c9640-958b-4387-87b3-ce636c47a372\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq" Jul 21 18:33:55.226962 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:55.226852 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq" Jul 21 18:33:55.570140 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:55.570120 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq"] Jul 21 18:33:55.572320 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:33:55.572294 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc96c9640_958b_4387_87b3_ce636c47a372.slice/crio-9e98edafdb77a621b0a9974f52adfcc48a54d101e95061e6adf2de3ca669096b WatchSource:0}: Error finding container 9e98edafdb77a621b0a9974f52adfcc48a54d101e95061e6adf2de3ca669096b: Status 404 returned error can't find the container with id 9e98edafdb77a621b0a9974f52adfcc48a54d101e95061e6adf2de3ca669096b Jul 21 18:33:56.526199 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:56.526104 2568 generic.go:356] "Generic (PLEG): container finished" podID="c96c9640-958b-4387-87b3-ce636c47a372" containerID="27b5306cb328cd408b537bdf597f3b94227111a0d282c3b025bb08d1fdaeb34f" exitCode=0 Jul 21 18:33:56.526579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:56.526191 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq" event={"ID":"c96c9640-958b-4387-87b3-ce636c47a372","Type":"ContainerDied","Data":"27b5306cb328cd408b537bdf597f3b94227111a0d282c3b025bb08d1fdaeb34f"} Jul 21 18:33:56.526579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:56.526286 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq" event={"ID":"c96c9640-958b-4387-87b3-ce636c47a372","Type":"ContainerStarted","Data":"9e98edafdb77a621b0a9974f52adfcc48a54d101e95061e6adf2de3ca669096b"} Jul 21 18:33:57.661535 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:57.661510 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq" Jul 21 18:33:57.703612 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:57.703575 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c96c9640-958b-4387-87b3-ce636c47a372-maas-api-service-ca\") pod \"c96c9640-958b-4387-87b3-ce636c47a372\" (UID: \"c96c9640-958b-4387-87b3-ce636c47a372\") " Jul 21 18:33:57.703946 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:57.703909 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c96c9640-958b-4387-87b3-ce636c47a372-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c96c9640-958b-4387-87b3-ce636c47a372" (UID: "c96c9640-958b-4387-87b3-ce636c47a372"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:33:57.804896 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:57.804808 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c96c9640-958b-4387-87b3-ce636c47a372-maas-api-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:33:58.538418 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:58.538384 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq" event={"ID":"c96c9640-958b-4387-87b3-ce636c47a372","Type":"ContainerDied","Data":"9e98edafdb77a621b0a9974f52adfcc48a54d101e95061e6adf2de3ca669096b"} Jul 21 18:33:58.538589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:58.538422 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq" Jul 21 18:33:58.538589 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:33:58.538432 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e98edafdb77a621b0a9974f52adfcc48a54d101e95061e6adf2de3ca669096b" Jul 21 18:34:09.128211 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:09.128168 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:34:10.468258 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.468223 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz"] Jul 21 18:34:10.468728 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.468501 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" podUID="c66188f9-6b68-44cc-994b-b7dce98883c2" containerName="maas-api" containerID="cri-o://b504288e0b4a0720fd1d5e69d3462789ea735cdf7f12f3ecce6084c6cdccf34a" gracePeriod=30 Jul 21 18:34:10.496878 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.496849 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:34:10.605347 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.605312 2568 generic.go:356] "Generic (PLEG): container finished" podID="c66188f9-6b68-44cc-994b-b7dce98883c2" containerID="b504288e0b4a0720fd1d5e69d3462789ea735cdf7f12f3ecce6084c6cdccf34a" exitCode=0 Jul 21 18:34:10.605538 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.605382 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" event={"ID":"c66188f9-6b68-44cc-994b-b7dce98883c2","Type":"ContainerDied","Data":"b504288e0b4a0720fd1d5e69d3462789ea735cdf7f12f3ecce6084c6cdccf34a"} Jul 21 18:34:10.720407 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.720341 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:34:10.824966 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.824901 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6qfw\" (UniqueName: \"kubernetes.io/projected/c66188f9-6b68-44cc-994b-b7dce98883c2-kube-api-access-v6qfw\") pod \"c66188f9-6b68-44cc-994b-b7dce98883c2\" (UID: \"c66188f9-6b68-44cc-994b-b7dce98883c2\") " Jul 21 18:34:10.824966 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.824966 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c66188f9-6b68-44cc-994b-b7dce98883c2-openshift-service-ca\") pod \"c66188f9-6b68-44cc-994b-b7dce98883c2\" (UID: \"c66188f9-6b68-44cc-994b-b7dce98883c2\") " Jul 21 18:34:10.825192 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.825039 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c66188f9-6b68-44cc-994b-b7dce98883c2-maas-api-tls\") pod \"c66188f9-6b68-44cc-994b-b7dce98883c2\" (UID: \"c66188f9-6b68-44cc-994b-b7dce98883c2\") " Jul 21 18:34:10.825393 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.825368 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c66188f9-6b68-44cc-994b-b7dce98883c2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c66188f9-6b68-44cc-994b-b7dce98883c2" (UID: "c66188f9-6b68-44cc-994b-b7dce98883c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:34:10.827091 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.827070 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c66188f9-6b68-44cc-994b-b7dce98883c2-kube-api-access-v6qfw" (OuterVolumeSpecName: "kube-api-access-v6qfw") pod "c66188f9-6b68-44cc-994b-b7dce98883c2" (UID: "c66188f9-6b68-44cc-994b-b7dce98883c2"). InnerVolumeSpecName "kube-api-access-v6qfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:34:10.827245 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.827175 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c66188f9-6b68-44cc-994b-b7dce98883c2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c66188f9-6b68-44cc-994b-b7dce98883c2" (UID: "c66188f9-6b68-44cc-994b-b7dce98883c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:34:10.926588 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.926546 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c66188f9-6b68-44cc-994b-b7dce98883c2-maas-api-tls\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:34:10.926588 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.926581 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6qfw\" (UniqueName: \"kubernetes.io/projected/c66188f9-6b68-44cc-994b-b7dce98883c2-kube-api-access-v6qfw\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:34:10.926588 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:10.926590 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c66188f9-6b68-44cc-994b-b7dce98883c2-openshift-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:34:11.612638 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:11.612607 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" Jul 21 18:34:11.613128 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:11.612641 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz" event={"ID":"c66188f9-6b68-44cc-994b-b7dce98883c2","Type":"ContainerDied","Data":"8ebbaf4596ae24fae7d06af6003834fe52c08d024e3cc37e5f7d5e589621043e"} Jul 21 18:34:11.613128 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:11.612698 2568 scope.go:117] "RemoveContainer" containerID="b504288e0b4a0720fd1d5e69d3462789ea735cdf7f12f3ecce6084c6cdccf34a" Jul 21 18:34:11.629399 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:11.629374 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz"] Jul 21 18:34:11.632970 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:11.632931 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-15fecbf7-6b9546f7f4-pzvwz"] Jul 21 18:34:13.219125 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:13.219092 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c66188f9-6b68-44cc-994b-b7dce98883c2" path="/var/lib/kubelet/pods/c66188f9-6b68-44cc-994b-b7dce98883c2/volumes" Jul 21 18:34:15.005155 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:15.005124 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:34:15.016263 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:15.016233 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq"] Jul 21 18:34:15.021669 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:15.021643 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-15fecbf7-7wqqq"] Jul 21 18:34:15.218621 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:15.218587 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c96c9640-958b-4387-87b3-ce636c47a372" path="/var/lib/kubelet/pods/c96c9640-958b-4387-87b3-ce636c47a372/volumes" Jul 21 18:34:17.242204 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:17.242172 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:34:19.331560 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:19.331521 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:34:22.270221 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:22.270136 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:34:32.636270 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:32.636231 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:34:53.849319 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:53.849285 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:34:58.618016 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:58.617976 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:34:59.311403 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:59.311378 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:34:59.315552 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:59.315521 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:34:59.317585 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:59.317560 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:34:59.321532 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:59.321511 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:34:59.503316 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:59.503283 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:34:59.863570 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:59.863537 2568 scope.go:117] "RemoveContainer" containerID="79e7e76f00fbfa5e9445dec791f33bc73de3963d268b034f04f23c500a8c1933" Jul 21 18:34:59.872069 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:34:59.872046 2568 scope.go:117] "RemoveContainer" containerID="0dad872019c123a1a70e1164abc0a863df30aca37ecc3c24a22b9fdd2a791d73" Jul 21 18:35:00.579676 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:00.579638 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:00.680145 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:00.680110 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:05.263506 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:05.263471 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:05.363162 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:05.363126 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:05.567444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:05.567362 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:05.814042 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:05.814013 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:06.874159 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:06.874119 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:06.971318 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:06.971282 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:07.069031 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:07.068998 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:07.920702 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:07.920662 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:08.463505 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:08.463467 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:09.468412 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:09.468376 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:35:09.637528 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:09.637495 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:15.447407 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:15.447370 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:15.509543 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:15.509509 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:16.146984 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:16.146922 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:22.215054 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:22.215017 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:22.241253 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:22.241220 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:35:22.314534 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:22.314497 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:22.413617 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:22.413587 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:22.514120 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:22.514032 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:32.960791 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:32.960752 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:33.312903 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:33.312819 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:54.298248 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:54.298166 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:35:59.900200 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:35:59.900169 2568 scope.go:117] "RemoveContainer" containerID="db5da826d417617283e189d27b742f99f6af88ee3774a7bd9d57bfb02a2f4287" Jul 21 18:36:00.836224 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:00.836189 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:00.938228 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:00.938194 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:01.195189 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:01.195158 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:01.555727 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:01.555626 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:02.247087 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:02.247051 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:04.698735 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.698691 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:04.782174 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.782144 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29"] Jul 21 18:36:04.782502 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.782489 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c66188f9-6b68-44cc-994b-b7dce98883c2" containerName="maas-api" Jul 21 18:36:04.782545 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.782504 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c66188f9-6b68-44cc-994b-b7dce98883c2" containerName="maas-api" Jul 21 18:36:04.782545 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.782521 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c96c9640-958b-4387-87b3-ce636c47a372" containerName="revoke-keys" Jul 21 18:36:04.782545 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.782526 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c96c9640-958b-4387-87b3-ce636c47a372" containerName="revoke-keys" Jul 21 18:36:04.782642 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.782577 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c96c9640-958b-4387-87b3-ce636c47a372" containerName="revoke-keys" Jul 21 18:36:04.782642 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.782586 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c66188f9-6b68-44cc-994b-b7dce98883c2" containerName="maas-api" Jul 21 18:36:04.793992 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.793960 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:36:04.796894 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.796866 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 18:36:04.797142 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.797126 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-4d82e2-serving-cert\"" Jul 21 18:36:04.797198 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.797139 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sh2jm\"" Jul 21 18:36:04.797487 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.797469 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 18:36:04.802565 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.802538 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:04.805991 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.805964 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29"] Jul 21 18:36:04.839225 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.839192 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29\" (UID: \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:36:04.839435 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.839234 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-maas-api-tls\") pod \"maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29\" (UID: \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:36:04.839435 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.839271 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvdjj\" (UniqueName: \"kubernetes.io/projected/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-kube-api-access-mvdjj\") pod \"maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29\" (UID: \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:36:04.939692 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.939653 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29\" (UID: \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:36:04.939879 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.939707 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-maas-api-tls\") pod \"maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29\" (UID: \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:36:04.939879 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.939740 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvdjj\" (UniqueName: \"kubernetes.io/projected/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-kube-api-access-mvdjj\") pod \"maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29\" (UID: \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:36:04.939879 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:36:04.939833 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-serving-cert: secret "maas-api-e2e-inf-a-4d82e2-serving-cert" not found Jul 21 18:36:04.940034 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:36:04.939899 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-maas-api-tls podName:a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423 nodeName:}" failed. No retries permitted until 2026-07-21 18:36:05.439882244 +0000 UTC m=+2166.708193663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-maas-api-tls") pod "maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" (UID: "a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423") : secret "maas-api-e2e-inf-a-4d82e2-serving-cert" not found Jul 21 18:36:04.940316 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.940298 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29\" (UID: \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:36:04.944842 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.944808 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:04.965407 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:04.965342 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvdjj\" (UniqueName: \"kubernetes.io/projected/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-kube-api-access-mvdjj\") pod \"maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29\" (UID: \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:36:05.079833 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:05.079797 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:05.443330 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:05.443296 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-maas-api-tls\") pod \"maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29\" (UID: \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:36:05.445773 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:05.445746 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-maas-api-tls\") pod \"maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29\" (UID: \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:36:05.633745 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:05.633704 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:05.706650 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:05.706551 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:36:05.735460 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:05.735431 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:05.847027 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:05.847000 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29"] Jul 21 18:36:05.849291 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:36:05.849267 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ac1b8e_ddb4_4bcc_a2f7_da48a6158423.slice/crio-54fb6b8e924e2b967e26d74e75eae405075fe27b66cec52530e0983edd24e445 WatchSource:0}: Error finding container 54fb6b8e924e2b967e26d74e75eae405075fe27b66cec52530e0983edd24e445: Status 404 returned error can't find the container with id 54fb6b8e924e2b967e26d74e75eae405075fe27b66cec52530e0983edd24e445 Jul 21 18:36:06.230832 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:06.230798 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" event={"ID":"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423","Type":"ContainerStarted","Data":"54fb6b8e924e2b967e26d74e75eae405075fe27b66cec52530e0983edd24e445"} Jul 21 18:36:07.236926 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:07.236881 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" event={"ID":"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423","Type":"ContainerStarted","Data":"dc43e15cdcbaef112ab80860e70bfc18fb497c86032b406319ee054407182f72"} Jul 21 18:36:07.237331 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:07.236990 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:36:07.255382 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:07.255331 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" podStartSLOduration=2.7084653210000003 podStartE2EDuration="3.255316605s" totalImagesPullingTime="546.851284ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:36:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:36:07.253340729 +0000 UTC m=+2168.521652169" watchObservedRunningTime="2026-07-21 18:36:07.255316605 +0000 UTC m=+2168.523628045" Jul 21 18:36:09.232614 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:09.232582 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:09.559336 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:09.559253 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:09.733053 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:09.733016 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:36:09.869294 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:09.869265 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:10.171038 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:10.170957 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:12.756336 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:12.756305 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:12.884597 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:12.884564 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:12.895899 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:12.895868 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf"] Jul 21 18:36:12.899721 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:12.899703 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:36:12.902896 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:12.902878 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-b2c46d-serving-cert\"" Jul 21 18:36:12.919385 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:12.919362 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf"] Jul 21 18:36:13.000382 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.000349 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/323af022-e05c-4792-bdef-3c0516f9c393-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf\" (UID: \"323af022-e05c-4792-bdef-3c0516f9c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:36:13.000586 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.000391 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/323af022-e05c-4792-bdef-3c0516f9c393-maas-api-tls\") pod \"maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf\" (UID: \"323af022-e05c-4792-bdef-3c0516f9c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:36:13.000586 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.000419 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndsnt\" (UniqueName: \"kubernetes.io/projected/323af022-e05c-4792-bdef-3c0516f9c393-kube-api-access-ndsnt\") pod \"maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf\" (UID: \"323af022-e05c-4792-bdef-3c0516f9c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:36:13.011656 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.011587 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:13.101902 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.101865 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/323af022-e05c-4792-bdef-3c0516f9c393-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf\" (UID: \"323af022-e05c-4792-bdef-3c0516f9c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:36:13.101902 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.101909 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/323af022-e05c-4792-bdef-3c0516f9c393-maas-api-tls\") pod \"maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf\" (UID: \"323af022-e05c-4792-bdef-3c0516f9c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:36:13.102148 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.101959 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndsnt\" (UniqueName: \"kubernetes.io/projected/323af022-e05c-4792-bdef-3c0516f9c393-kube-api-access-ndsnt\") pod \"maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf\" (UID: \"323af022-e05c-4792-bdef-3c0516f9c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:36:13.102515 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.102494 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/323af022-e05c-4792-bdef-3c0516f9c393-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf\" (UID: \"323af022-e05c-4792-bdef-3c0516f9c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:36:13.104342 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.104322 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/323af022-e05c-4792-bdef-3c0516f9c393-maas-api-tls\") pod \"maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf\" (UID: \"323af022-e05c-4792-bdef-3c0516f9c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:36:13.108292 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.108271 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:13.112606 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.112585 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndsnt\" (UniqueName: \"kubernetes.io/projected/323af022-e05c-4792-bdef-3c0516f9c393-kube-api-access-ndsnt\") pod \"maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf\" (UID: \"323af022-e05c-4792-bdef-3c0516f9c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:36:13.209444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.209403 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:36:13.247880 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.247855 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:36:13.353462 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.353433 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf"] Jul 21 18:36:13.355123 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:36:13.355092 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod323af022_e05c_4792_bdef_3c0516f9c393.slice/crio-bbbf4606069bcfcf8e20ba24e6d75c7a0f569f1ca32ef137692f48c2bc2731da WatchSource:0}: Error finding container bbbf4606069bcfcf8e20ba24e6d75c7a0f569f1ca32ef137692f48c2bc2731da: Status 404 returned error can't find the container with id bbbf4606069bcfcf8e20ba24e6d75c7a0f569f1ca32ef137692f48c2bc2731da Jul 21 18:36:13.549200 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.549118 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:13.650248 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:13.650213 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:14.276746 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:14.276709 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" event={"ID":"323af022-e05c-4792-bdef-3c0516f9c393","Type":"ContainerStarted","Data":"22d4056ea4ae03fd0cda02bdbe89450c8bf552e18f2208d499bdfac7cb6545e1"} Jul 21 18:36:14.277213 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:14.276760 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" event={"ID":"323af022-e05c-4792-bdef-3c0516f9c393","Type":"ContainerStarted","Data":"bbbf4606069bcfcf8e20ba24e6d75c7a0f569f1ca32ef137692f48c2bc2731da"} Jul 21 18:36:14.277213 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:14.276817 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:36:14.294926 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:14.294877 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" podStartSLOduration=1.925575783 podStartE2EDuration="2.294862763s" totalImagesPullingTime="369.28698ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:36:14.292996004 +0000 UTC m=+2175.561307447" watchObservedRunningTime="2026-07-21 18:36:14.294862763 +0000 UTC m=+2175.563174205" Jul 21 18:36:17.226010 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:17.225979 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:18.930348 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:18.930312 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:36:20.287549 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:20.287515 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:36:26.556287 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:26.556254 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:26.658914 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:26.658871 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:35.542815 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:35.542777 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:35.644154 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:35.644121 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:36.251798 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:36.251758 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:39.744654 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:39.744618 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:36:41.903084 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:41.903045 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:42.018040 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:42.018008 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:52.245429 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:52.245391 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:36:58.734417 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:58.734375 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:36:59.921045 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:59.921011 2568 scope.go:117] "RemoveContainer" containerID="2c7b61c8223a75fb0bbb5b30b54b7c7e551d0849cdc68a1f1ad9e4e1f08b1b1c" Jul 21 18:36:59.931800 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:36:59.931781 2568 scope.go:117] "RemoveContainer" containerID="3faaf0fe2f255cd186e4c412c36dc2ee8e77756a713298374537ecaebeb35808" Jul 21 18:37:04.240126 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:04.240082 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:37:47.290916 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:47.290824 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:37:48.252801 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:48.252760 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:37:48.341827 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:48.341790 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7"] Jul 21 18:37:48.345611 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:48.345587 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7" Jul 21 18:37:48.348051 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:48.348030 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:37:48.353179 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:48.353111 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7"] Jul 21 18:37:48.431469 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:48.431433 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15a692ad-b144-4d80-ba0c-099e20d05068-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7\" (UID: \"15a692ad-b144-4d80-ba0c-099e20d05068\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7" Jul 21 18:37:48.532488 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:48.532394 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15a692ad-b144-4d80-ba0c-099e20d05068-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7\" (UID: \"15a692ad-b144-4d80-ba0c-099e20d05068\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7" Jul 21 18:37:48.533015 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:48.532996 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15a692ad-b144-4d80-ba0c-099e20d05068-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7\" (UID: \"15a692ad-b144-4d80-ba0c-099e20d05068\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7" Jul 21 18:37:48.656611 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:48.656568 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7" Jul 21 18:37:48.800885 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:48.800858 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7"] Jul 21 18:37:48.802683 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:37:48.802648 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a692ad_b144_4d80_ba0c_099e20d05068.slice/crio-c50997db6db51fbcb44bbca8d8e1de73a4c7ef929ee1afe8338e6129831368cf WatchSource:0}: Error finding container c50997db6db51fbcb44bbca8d8e1de73a4c7ef929ee1afe8338e6129831368cf: Status 404 returned error can't find the container with id c50997db6db51fbcb44bbca8d8e1de73a4c7ef929ee1afe8338e6129831368cf Jul 21 18:37:49.064964 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:49.064850 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:37:49.164482 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:49.164449 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:37:49.784739 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:49.784702 2568 generic.go:356] "Generic (PLEG): container finished" podID="15a692ad-b144-4d80-ba0c-099e20d05068" containerID="ba7878cf7d5d4bc9b87af084aa837befb5c8cb6504c1f69e1b51b86921d1f367" exitCode=0 Jul 21 18:37:49.785171 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:49.784796 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7" event={"ID":"15a692ad-b144-4d80-ba0c-099e20d05068","Type":"ContainerDied","Data":"ba7878cf7d5d4bc9b87af084aa837befb5c8cb6504c1f69e1b51b86921d1f367"} Jul 21 18:37:49.785171 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:49.784868 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7" event={"ID":"15a692ad-b144-4d80-ba0c-099e20d05068","Type":"ContainerStarted","Data":"c50997db6db51fbcb44bbca8d8e1de73a4c7ef929ee1afe8338e6129831368cf"} Jul 21 18:37:50.923353 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:50.923325 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7" Jul 21 18:37:51.053745 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:51.053653 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15a692ad-b144-4d80-ba0c-099e20d05068-maas-api-service-ca\") pod \"15a692ad-b144-4d80-ba0c-099e20d05068\" (UID: \"15a692ad-b144-4d80-ba0c-099e20d05068\") " Jul 21 18:37:51.054071 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:51.054044 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15a692ad-b144-4d80-ba0c-099e20d05068-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "15a692ad-b144-4d80-ba0c-099e20d05068" (UID: "15a692ad-b144-4d80-ba0c-099e20d05068"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:37:51.154540 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:51.154498 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15a692ad-b144-4d80-ba0c-099e20d05068-maas-api-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:37:51.354284 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:51.354247 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:37:51.796486 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:51.796397 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7" Jul 21 18:37:51.796486 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:51.796402 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7" event={"ID":"15a692ad-b144-4d80-ba0c-099e20d05068","Type":"ContainerDied","Data":"c50997db6db51fbcb44bbca8d8e1de73a4c7ef929ee1afe8338e6129831368cf"} Jul 21 18:37:51.796714 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:51.796517 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c50997db6db51fbcb44bbca8d8e1de73a4c7ef929ee1afe8338e6129831368cf" Jul 21 18:37:59.059141 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.059103 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29"] Jul 21 18:37:59.059538 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.059329 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" podUID="a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423" containerName="maas-api" containerID="cri-o://dc43e15cdcbaef112ab80860e70bfc18fb497c86032b406319ee054407182f72" gracePeriod=30 Jul 21 18:37:59.128714 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.128664 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:37:59.304497 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.304469 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:37:59.429874 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.429840 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-maas-api-tls\") pod \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\" (UID: \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\") " Jul 21 18:37:59.430106 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.429896 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-openshift-service-ca\") pod \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\" (UID: \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\") " Jul 21 18:37:59.430106 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.429928 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvdjj\" (UniqueName: \"kubernetes.io/projected/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-kube-api-access-mvdjj\") pod \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\" (UID: \"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423\") " Jul 21 18:37:59.430302 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.430278 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423" (UID: "a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:37:59.432058 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.432036 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423" (UID: "a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:37:59.432130 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.432070 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-kube-api-access-mvdjj" (OuterVolumeSpecName: "kube-api-access-mvdjj") pod "a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423" (UID: "a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423"). InnerVolumeSpecName "kube-api-access-mvdjj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:37:59.531183 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.531129 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-maas-api-tls\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:37:59.531183 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.531178 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-openshift-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:37:59.531183 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.531190 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvdjj\" (UniqueName: \"kubernetes.io/projected/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423-kube-api-access-mvdjj\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:37:59.846222 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.846190 2568 generic.go:356] "Generic (PLEG): container finished" podID="a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423" containerID="dc43e15cdcbaef112ab80860e70bfc18fb497c86032b406319ee054407182f72" exitCode=0 Jul 21 18:37:59.846389 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.846233 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" Jul 21 18:37:59.846389 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.846279 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" event={"ID":"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423","Type":"ContainerDied","Data":"dc43e15cdcbaef112ab80860e70bfc18fb497c86032b406319ee054407182f72"} Jul 21 18:37:59.846389 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.846324 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29" event={"ID":"a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423","Type":"ContainerDied","Data":"54fb6b8e924e2b967e26d74e75eae405075fe27b66cec52530e0983edd24e445"} Jul 21 18:37:59.846389 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.846344 2568 scope.go:117] "RemoveContainer" containerID="dc43e15cdcbaef112ab80860e70bfc18fb497c86032b406319ee054407182f72" Jul 21 18:37:59.856067 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.856047 2568 scope.go:117] "RemoveContainer" containerID="dc43e15cdcbaef112ab80860e70bfc18fb497c86032b406319ee054407182f72" Jul 21 18:37:59.857814 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:37:59.857784 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc43e15cdcbaef112ab80860e70bfc18fb497c86032b406319ee054407182f72\": container with ID starting with dc43e15cdcbaef112ab80860e70bfc18fb497c86032b406319ee054407182f72 not found: ID does not exist" containerID="dc43e15cdcbaef112ab80860e70bfc18fb497c86032b406319ee054407182f72" Jul 21 18:37:59.857915 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.857821 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc43e15cdcbaef112ab80860e70bfc18fb497c86032b406319ee054407182f72"} err="failed to get container status \"dc43e15cdcbaef112ab80860e70bfc18fb497c86032b406319ee054407182f72\": rpc error: code = NotFound desc = could not find container \"dc43e15cdcbaef112ab80860e70bfc18fb497c86032b406319ee054407182f72\": container with ID starting with dc43e15cdcbaef112ab80860e70bfc18fb497c86032b406319ee054407182f72 not found: ID does not exist" Jul 21 18:37:59.873164 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.873137 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29"] Jul 21 18:37:59.883708 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:37:59.883679 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4d82e2-7984f58fc6-hhg29"] Jul 21 18:38:01.218485 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:01.218448 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423" path="/var/lib/kubelet/pods/a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423/volumes" Jul 21 18:38:03.434701 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:03.434664 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:03.459192 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:03.459158 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7"] Jul 21 18:38:03.462026 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:03.462001 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4d82e2-gzjj7"] Jul 21 18:38:03.867887 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:03.867851 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:03.940591 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:03.940555 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:38:05.219531 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:05.219494 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15a692ad-b144-4d80-ba0c-099e20d05068" path="/var/lib/kubelet/pods/15a692ad-b144-4d80-ba0c-099e20d05068/volumes" Jul 21 18:38:11.354825 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:11.354789 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:11.540710 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:11.540668 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:11.779472 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:11.779388 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:12.004005 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.003958 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:12.153117 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.153075 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv"] Jul 21 18:38:12.153438 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.153427 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15a692ad-b144-4d80-ba0c-099e20d05068" containerName="revoke-keys" Jul 21 18:38:12.153484 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.153441 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="15a692ad-b144-4d80-ba0c-099e20d05068" containerName="revoke-keys" Jul 21 18:38:12.153484 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.153453 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423" containerName="maas-api" Jul 21 18:38:12.153484 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.153459 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423" containerName="maas-api" Jul 21 18:38:12.153574 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.153508 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1ac1b8e-ddb4-4bcc-a2f7-da48a6158423" containerName="maas-api" Jul 21 18:38:12.153574 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.153517 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="15a692ad-b144-4d80-ba0c-099e20d05068" containerName="revoke-keys" Jul 21 18:38:12.156738 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.156717 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv" Jul 21 18:38:12.161128 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.161098 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:38:12.176177 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.176133 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv"] Jul 21 18:38:12.241743 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.241704 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24d29c92-9fc1-43e5-ad53-1a28673d9cad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv\" (UID: \"24d29c92-9fc1-43e5-ad53-1a28673d9cad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv" Jul 21 18:38:12.343145 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.343107 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24d29c92-9fc1-43e5-ad53-1a28673d9cad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv\" (UID: \"24d29c92-9fc1-43e5-ad53-1a28673d9cad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv" Jul 21 18:38:12.343809 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.343790 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24d29c92-9fc1-43e5-ad53-1a28673d9cad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv\" (UID: \"24d29c92-9fc1-43e5-ad53-1a28673d9cad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv" Jul 21 18:38:12.467842 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.467751 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv" Jul 21 18:38:12.644413 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.644380 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv"] Jul 21 18:38:12.647206 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:38:12.647175 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24d29c92_9fc1_43e5_ad53_1a28673d9cad.slice/crio-ba28240c707329d51f75ca6e9fb8ed2255874f5360e8e71b135a8d9a4fb8f48f WatchSource:0}: Error finding container ba28240c707329d51f75ca6e9fb8ed2255874f5360e8e71b135a8d9a4fb8f48f: Status 404 returned error can't find the container with id ba28240c707329d51f75ca6e9fb8ed2255874f5360e8e71b135a8d9a4fb8f48f Jul 21 18:38:12.914873 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:12.914836 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv" event={"ID":"24d29c92-9fc1-43e5-ad53-1a28673d9cad","Type":"ContainerStarted","Data":"ba28240c707329d51f75ca6e9fb8ed2255874f5360e8e71b135a8d9a4fb8f48f"} Jul 21 18:38:13.248793 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:13.248694 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:13.372689 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:13.372652 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:13.921686 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:13.921652 2568 generic.go:356] "Generic (PLEG): container finished" podID="24d29c92-9fc1-43e5-ad53-1a28673d9cad" containerID="be885deab41681001cd7c7e00f4104ffe97bd4a3050f623265f48b3db1ba30a6" exitCode=0 Jul 21 18:38:13.922099 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:13.921739 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv" event={"ID":"24d29c92-9fc1-43e5-ad53-1a28673d9cad","Type":"ContainerDied","Data":"be885deab41681001cd7c7e00f4104ffe97bd4a3050f623265f48b3db1ba30a6"} Jul 21 18:38:15.051743 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:15.051715 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv" Jul 21 18:38:15.065702 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:15.065673 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24d29c92-9fc1-43e5-ad53-1a28673d9cad-maas-api-service-ca\") pod \"24d29c92-9fc1-43e5-ad53-1a28673d9cad\" (UID: \"24d29c92-9fc1-43e5-ad53-1a28673d9cad\") " Jul 21 18:38:15.066083 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:15.066061 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24d29c92-9fc1-43e5-ad53-1a28673d9cad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "24d29c92-9fc1-43e5-ad53-1a28673d9cad" (UID: "24d29c92-9fc1-43e5-ad53-1a28673d9cad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:38:15.167070 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:15.167024 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24d29c92-9fc1-43e5-ad53-1a28673d9cad-maas-api-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:38:15.933136 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:15.933107 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv" Jul 21 18:38:15.933136 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:15.933118 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv" event={"ID":"24d29c92-9fc1-43e5-ad53-1a28673d9cad","Type":"ContainerDied","Data":"ba28240c707329d51f75ca6e9fb8ed2255874f5360e8e71b135a8d9a4fb8f48f"} Jul 21 18:38:15.933346 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:15.933160 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba28240c707329d51f75ca6e9fb8ed2255874f5360e8e71b135a8d9a4fb8f48f" Jul 21 18:38:23.315889 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.315846 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf"] Jul 21 18:38:23.316409 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.316107 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" podUID="323af022-e05c-4792-bdef-3c0516f9c393" containerName="maas-api" containerID="cri-o://22d4056ea4ae03fd0cda02bdbe89450c8bf552e18f2208d499bdfac7cb6545e1" gracePeriod=30 Jul 21 18:38:23.377108 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.377069 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:23.567259 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.567182 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:38:23.636710 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.636670 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/323af022-e05c-4792-bdef-3c0516f9c393-openshift-service-ca\") pod \"323af022-e05c-4792-bdef-3c0516f9c393\" (UID: \"323af022-e05c-4792-bdef-3c0516f9c393\") " Jul 21 18:38:23.636710 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.636711 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/323af022-e05c-4792-bdef-3c0516f9c393-maas-api-tls\") pod \"323af022-e05c-4792-bdef-3c0516f9c393\" (UID: \"323af022-e05c-4792-bdef-3c0516f9c393\") " Jul 21 18:38:23.637006 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.636748 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndsnt\" (UniqueName: \"kubernetes.io/projected/323af022-e05c-4792-bdef-3c0516f9c393-kube-api-access-ndsnt\") pod \"323af022-e05c-4792-bdef-3c0516f9c393\" (UID: \"323af022-e05c-4792-bdef-3c0516f9c393\") " Jul 21 18:38:23.637077 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.637048 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/323af022-e05c-4792-bdef-3c0516f9c393-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "323af022-e05c-4792-bdef-3c0516f9c393" (UID: "323af022-e05c-4792-bdef-3c0516f9c393"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:38:23.638868 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.638845 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/323af022-e05c-4792-bdef-3c0516f9c393-kube-api-access-ndsnt" (OuterVolumeSpecName: "kube-api-access-ndsnt") pod "323af022-e05c-4792-bdef-3c0516f9c393" (UID: "323af022-e05c-4792-bdef-3c0516f9c393"). InnerVolumeSpecName "kube-api-access-ndsnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:38:23.638966 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.638868 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/323af022-e05c-4792-bdef-3c0516f9c393-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "323af022-e05c-4792-bdef-3c0516f9c393" (UID: "323af022-e05c-4792-bdef-3c0516f9c393"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:38:23.737855 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.737817 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/323af022-e05c-4792-bdef-3c0516f9c393-maas-api-tls\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:38:23.737855 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.737852 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndsnt\" (UniqueName: \"kubernetes.io/projected/323af022-e05c-4792-bdef-3c0516f9c393-kube-api-access-ndsnt\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:38:23.737855 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.737862 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/323af022-e05c-4792-bdef-3c0516f9c393-openshift-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:38:23.975639 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.975603 2568 generic.go:356] "Generic (PLEG): container finished" podID="323af022-e05c-4792-bdef-3c0516f9c393" containerID="22d4056ea4ae03fd0cda02bdbe89450c8bf552e18f2208d499bdfac7cb6545e1" exitCode=0 Jul 21 18:38:23.975847 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.975648 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" Jul 21 18:38:23.975847 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.975676 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" event={"ID":"323af022-e05c-4792-bdef-3c0516f9c393","Type":"ContainerDied","Data":"22d4056ea4ae03fd0cda02bdbe89450c8bf552e18f2208d499bdfac7cb6545e1"} Jul 21 18:38:23.975847 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.975729 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf" event={"ID":"323af022-e05c-4792-bdef-3c0516f9c393","Type":"ContainerDied","Data":"bbbf4606069bcfcf8e20ba24e6d75c7a0f569f1ca32ef137692f48c2bc2731da"} Jul 21 18:38:23.975847 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.975756 2568 scope.go:117] "RemoveContainer" containerID="22d4056ea4ae03fd0cda02bdbe89450c8bf552e18f2208d499bdfac7cb6545e1" Jul 21 18:38:23.985008 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.984959 2568 scope.go:117] "RemoveContainer" containerID="22d4056ea4ae03fd0cda02bdbe89450c8bf552e18f2208d499bdfac7cb6545e1" Jul 21 18:38:23.985248 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:38:23.985231 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22d4056ea4ae03fd0cda02bdbe89450c8bf552e18f2208d499bdfac7cb6545e1\": container with ID starting with 22d4056ea4ae03fd0cda02bdbe89450c8bf552e18f2208d499bdfac7cb6545e1 not found: ID does not exist" containerID="22d4056ea4ae03fd0cda02bdbe89450c8bf552e18f2208d499bdfac7cb6545e1" Jul 21 18:38:23.985294 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:23.985256 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22d4056ea4ae03fd0cda02bdbe89450c8bf552e18f2208d499bdfac7cb6545e1"} err="failed to get container status \"22d4056ea4ae03fd0cda02bdbe89450c8bf552e18f2208d499bdfac7cb6545e1\": rpc error: code = NotFound desc = could not find container \"22d4056ea4ae03fd0cda02bdbe89450c8bf552e18f2208d499bdfac7cb6545e1\": container with ID starting with 22d4056ea4ae03fd0cda02bdbe89450c8bf552e18f2208d499bdfac7cb6545e1 not found: ID does not exist" Jul 21 18:38:24.016289 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:24.016251 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf"] Jul 21 18:38:24.024909 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:24.024868 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2c46d-5599b546dc-t4klf"] Jul 21 18:38:25.220460 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:25.220426 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="323af022-e05c-4792-bdef-3c0516f9c393" path="/var/lib/kubelet/pods/323af022-e05c-4792-bdef-3c0516f9c393/volumes" Jul 21 18:38:26.360988 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:26.360953 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:38:27.388038 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:27.388005 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:27.401632 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:27.401599 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv"] Jul 21 18:38:27.413754 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:27.413720 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2c46d-9l6lv"] Jul 21 18:38:27.819443 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:27.819351 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:29.219844 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:29.219806 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24d29c92-9fc1-43e5-ad53-1a28673d9cad" path="/var/lib/kubelet/pods/24d29c92-9fc1-43e5-ad53-1a28673d9cad/volumes" Jul 21 18:38:35.728487 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:35.728453 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:35.918986 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:35.918926 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:36.092138 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:36.092102 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:36.312159 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:36.312128 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:36.667147 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:36.667109 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:37.120679 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:37.120643 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:37.591265 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:37.591237 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:40.888261 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.888230 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:40.934326 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.934286 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5"] Jul 21 18:38:40.934699 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.934682 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24d29c92-9fc1-43e5-ad53-1a28673d9cad" containerName="revoke-keys" Jul 21 18:38:40.934779 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.934703 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="24d29c92-9fc1-43e5-ad53-1a28673d9cad" containerName="revoke-keys" Jul 21 18:38:40.934779 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.934719 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="323af022-e05c-4792-bdef-3c0516f9c393" containerName="maas-api" Jul 21 18:38:40.934779 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.934727 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="323af022-e05c-4792-bdef-3c0516f9c393" containerName="maas-api" Jul 21 18:38:40.934954 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.934819 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="323af022-e05c-4792-bdef-3c0516f9c393" containerName="maas-api" Jul 21 18:38:40.934954 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.934834 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="24d29c92-9fc1-43e5-ad53-1a28673d9cad" containerName="revoke-keys" Jul 21 18:38:40.937981 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.937957 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:38:40.946407 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.943917 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-9c004f-serving-cert\"" Jul 21 18:38:40.946407 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.944647 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 18:38:40.946407 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.945018 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 18:38:40.946407 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.945273 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sh2jm\"" Jul 21 18:38:40.974692 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.974658 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5"] Jul 21 18:38:40.994140 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.994105 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e95eb4ad-526e-4f09-8042-f76b78860e5c-maas-api-tls\") pod \"maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5\" (UID: \"e95eb4ad-526e-4f09-8042-f76b78860e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:38:40.994355 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.994152 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b26r\" (UniqueName: \"kubernetes.io/projected/e95eb4ad-526e-4f09-8042-f76b78860e5c-kube-api-access-5b26r\") pod \"maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5\" (UID: \"e95eb4ad-526e-4f09-8042-f76b78860e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:38:40.994355 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:40.994277 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e95eb4ad-526e-4f09-8042-f76b78860e5c-openshift-service-ca\") pod \"maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5\" (UID: \"e95eb4ad-526e-4f09-8042-f76b78860e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:38:41.045299 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:41.045264 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:41.095480 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:41.095439 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e95eb4ad-526e-4f09-8042-f76b78860e5c-maas-api-tls\") pod \"maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5\" (UID: \"e95eb4ad-526e-4f09-8042-f76b78860e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:38:41.095677 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:41.095493 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b26r\" (UniqueName: \"kubernetes.io/projected/e95eb4ad-526e-4f09-8042-f76b78860e5c-kube-api-access-5b26r\") pod \"maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5\" (UID: \"e95eb4ad-526e-4f09-8042-f76b78860e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:38:41.095677 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:38:41.095577 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-serving-cert: secret "maas-api-e2e-api-a-9c004f-serving-cert" not found Jul 21 18:38:41.095677 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:41.095593 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e95eb4ad-526e-4f09-8042-f76b78860e5c-openshift-service-ca\") pod \"maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5\" (UID: \"e95eb4ad-526e-4f09-8042-f76b78860e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:38:41.095677 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:38:41.095657 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95eb4ad-526e-4f09-8042-f76b78860e5c-maas-api-tls podName:e95eb4ad-526e-4f09-8042-f76b78860e5c nodeName:}" failed. No retries permitted until 2026-07-21 18:38:41.595637317 +0000 UTC m=+2322.863948754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e95eb4ad-526e-4f09-8042-f76b78860e5c-maas-api-tls") pod "maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" (UID: "e95eb4ad-526e-4f09-8042-f76b78860e5c") : secret "maas-api-e2e-api-a-9c004f-serving-cert" not found Jul 21 18:38:41.096180 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:41.096162 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e95eb4ad-526e-4f09-8042-f76b78860e5c-openshift-service-ca\") pod \"maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5\" (UID: \"e95eb4ad-526e-4f09-8042-f76b78860e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:38:41.114842 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:41.114807 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b26r\" (UniqueName: \"kubernetes.io/projected/e95eb4ad-526e-4f09-8042-f76b78860e5c-kube-api-access-5b26r\") pod \"maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5\" (UID: \"e95eb4ad-526e-4f09-8042-f76b78860e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:38:41.383073 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:41.383036 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:41.601567 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:41.601525 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e95eb4ad-526e-4f09-8042-f76b78860e5c-maas-api-tls\") pod \"maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5\" (UID: \"e95eb4ad-526e-4f09-8042-f76b78860e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:38:41.603927 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:41.603896 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e95eb4ad-526e-4f09-8042-f76b78860e5c-maas-api-tls\") pod \"maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5\" (UID: \"e95eb4ad-526e-4f09-8042-f76b78860e5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:38:41.851819 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:41.851782 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:38:41.906884 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:41.906840 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:42.004771 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:42.004743 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5"] Jul 21 18:38:42.006468 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:38:42.006443 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode95eb4ad_526e_4f09_8042_f76b78860e5c.slice/crio-77d517733ba6d5a39f7543635069b7f6377c7e3d885f6f4529f824da9ce8ce0e WatchSource:0}: Error finding container 77d517733ba6d5a39f7543635069b7f6377c7e3d885f6f4529f824da9ce8ce0e: Status 404 returned error can't find the container with id 77d517733ba6d5a39f7543635069b7f6377c7e3d885f6f4529f824da9ce8ce0e Jul 21 18:38:42.007746 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:42.007726 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 18:38:42.044409 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:42.044370 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:42.081982 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:42.081946 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" event={"ID":"e95eb4ad-526e-4f09-8042-f76b78860e5c","Type":"ContainerStarted","Data":"77d517733ba6d5a39f7543635069b7f6377c7e3d885f6f4529f824da9ce8ce0e"} Jul 21 18:38:43.088710 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:43.088670 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" event={"ID":"e95eb4ad-526e-4f09-8042-f76b78860e5c","Type":"ContainerStarted","Data":"c64c9a41feab619522b0e22c296798f9ed8889a1843b4d6cd7a6438b7a29dd66"} Jul 21 18:38:43.089116 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:43.088755 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:38:43.115302 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:43.115249 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" podStartSLOduration=2.248454957 podStartE2EDuration="3.115231175s" totalImagesPullingTime="866.776218ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:38:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:38:43.113477329 +0000 UTC m=+2324.381788771" watchObservedRunningTime="2026-07-21 18:38:43.115231175 +0000 UTC m=+2324.383542616" Jul 21 18:38:45.289956 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:45.289880 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:45.747853 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:45.747807 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:38:45.776167 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:45.776129 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:46.153156 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:46.153118 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:46.291271 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:46.291231 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:49.100274 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.100238 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:38:49.664766 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.664719 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:49.797714 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.797674 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz"] Jul 21 18:38:49.802440 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.802413 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:38:49.804930 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.804905 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-057ec0-serving-cert\"" Jul 21 18:38:49.819446 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.819420 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz"] Jul 21 18:38:49.872988 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.872930 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b67586-2374-4bb1-82aa-e6aab1870048-openshift-service-ca\") pod \"maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz\" (UID: \"13b67586-2374-4bb1-82aa-e6aab1870048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:38:49.872988 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.872993 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b67586-2374-4bb1-82aa-e6aab1870048-maas-api-tls\") pod \"maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz\" (UID: \"13b67586-2374-4bb1-82aa-e6aab1870048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:38:49.873229 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.873028 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6xmj\" (UniqueName: \"kubernetes.io/projected/13b67586-2374-4bb1-82aa-e6aab1870048-kube-api-access-x6xmj\") pod \"maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz\" (UID: \"13b67586-2374-4bb1-82aa-e6aab1870048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:38:49.883806 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.883780 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:49.973740 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.973643 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b67586-2374-4bb1-82aa-e6aab1870048-openshift-service-ca\") pod \"maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz\" (UID: \"13b67586-2374-4bb1-82aa-e6aab1870048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:38:49.973740 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.973694 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b67586-2374-4bb1-82aa-e6aab1870048-maas-api-tls\") pod \"maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz\" (UID: \"13b67586-2374-4bb1-82aa-e6aab1870048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:38:49.973740 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.973741 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6xmj\" (UniqueName: \"kubernetes.io/projected/13b67586-2374-4bb1-82aa-e6aab1870048-kube-api-access-x6xmj\") pod \"maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz\" (UID: \"13b67586-2374-4bb1-82aa-e6aab1870048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:38:49.974117 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:38:49.973809 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-serving-cert: secret "maas-api-e2e-api-b-057ec0-serving-cert" not found Jul 21 18:38:49.974117 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:38:49.973887 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13b67586-2374-4bb1-82aa-e6aab1870048-maas-api-tls podName:13b67586-2374-4bb1-82aa-e6aab1870048 nodeName:}" failed. No retries permitted until 2026-07-21 18:38:50.473867124 +0000 UTC m=+2331.742178546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/13b67586-2374-4bb1-82aa-e6aab1870048-maas-api-tls") pod "maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" (UID: "13b67586-2374-4bb1-82aa-e6aab1870048") : secret "maas-api-e2e-api-b-057ec0-serving-cert" not found Jul 21 18:38:49.974445 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.974423 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b67586-2374-4bb1-82aa-e6aab1870048-openshift-service-ca\") pod \"maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz\" (UID: \"13b67586-2374-4bb1-82aa-e6aab1870048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:38:49.983843 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.983820 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6xmj\" (UniqueName: \"kubernetes.io/projected/13b67586-2374-4bb1-82aa-e6aab1870048-kube-api-access-x6xmj\") pod \"maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz\" (UID: \"13b67586-2374-4bb1-82aa-e6aab1870048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:38:49.989479 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:49.989453 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:50.348988 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:50.348909 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:50.477721 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:50.477681 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b67586-2374-4bb1-82aa-e6aab1870048-maas-api-tls\") pod \"maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz\" (UID: \"13b67586-2374-4bb1-82aa-e6aab1870048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:38:50.480244 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:50.480213 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b67586-2374-4bb1-82aa-e6aab1870048-maas-api-tls\") pod \"maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz\" (UID: \"13b67586-2374-4bb1-82aa-e6aab1870048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:38:50.671570 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:50.671480 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:50.713016 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:50.712982 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:38:50.777493 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:50.777376 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:50.879375 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:50.879346 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz"] Jul 21 18:38:50.880786 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:38:50.880761 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b67586_2374_4bb1_82aa_e6aab1870048.slice/crio-720d166cae3599aa184ac19265fd3c63f77a45411d754e0cb9eca69cab37db5f WatchSource:0}: Error finding container 720d166cae3599aa184ac19265fd3c63f77a45411d754e0cb9eca69cab37db5f: Status 404 returned error can't find the container with id 720d166cae3599aa184ac19265fd3c63f77a45411d754e0cb9eca69cab37db5f Jul 21 18:38:51.134363 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:51.134327 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" event={"ID":"13b67586-2374-4bb1-82aa-e6aab1870048","Type":"ContainerStarted","Data":"720d166cae3599aa184ac19265fd3c63f77a45411d754e0cb9eca69cab37db5f"} Jul 21 18:38:52.145284 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:52.145231 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" event={"ID":"13b67586-2374-4bb1-82aa-e6aab1870048","Type":"ContainerStarted","Data":"0180066acde8bb08868d4837f36e6eee40f090674afb3dd4c2ffcc5d80384fd9"} Jul 21 18:38:52.145921 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:52.145899 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:38:52.167918 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:52.167858 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" podStartSLOduration=2.644869151 podStartE2EDuration="3.167838975s" totalImagesPullingTime="522.969824ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:38:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:38:52.166510855 +0000 UTC m=+2333.434822319" watchObservedRunningTime="2026-07-21 18:38:52.167838975 +0000 UTC m=+2333.436150419" Jul 21 18:38:54.192141 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:54.192109 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:38:57.752523 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:57.752488 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:38:58.156129 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:58.156100 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:38:59.967907 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:38:59.967873 2568 scope.go:117] "RemoveContainer" containerID="87e7a977f65e5929c71a745baa153a6b997a772ad6b2d29a329a040ae66e83b7" Jul 21 18:39:02.670860 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:02.670824 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf"] Jul 21 18:39:02.675488 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:02.675467 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf" Jul 21 18:39:02.678233 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:02.678211 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:39:02.682421 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:02.682397 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf"] Jul 21 18:39:02.788545 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:02.788497 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c03324c-7392-442d-bdd8-013c1debe6af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-9c004f-sjczf\" (UID: \"4c03324c-7392-442d-bdd8-013c1debe6af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf" Jul 21 18:39:02.889358 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:02.889321 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c03324c-7392-442d-bdd8-013c1debe6af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-9c004f-sjczf\" (UID: \"4c03324c-7392-442d-bdd8-013c1debe6af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf" Jul 21 18:39:02.890025 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:02.890001 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c03324c-7392-442d-bdd8-013c1debe6af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-9c004f-sjczf\" (UID: \"4c03324c-7392-442d-bdd8-013c1debe6af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf" Jul 21 18:39:02.975860 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:02.975771 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:02.987715 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:02.987691 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf" Jul 21 18:39:03.063921 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:03.063884 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:03.126556 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:03.126527 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf"] Jul 21 18:39:03.128168 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:39:03.128114 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c03324c_7392_442d_bdd8_013c1debe6af.slice/crio-0ca3baaac220fc90e69e3c87c8270494e7a03e1ee1d1db7d78737202dab64916 WatchSource:0}: Error finding container 0ca3baaac220fc90e69e3c87c8270494e7a03e1ee1d1db7d78737202dab64916: Status 404 returned error can't find the container with id 0ca3baaac220fc90e69e3c87c8270494e7a03e1ee1d1db7d78737202dab64916 Jul 21 18:39:03.206062 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:03.206023 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf" event={"ID":"4c03324c-7392-442d-bdd8-013c1debe6af","Type":"ContainerStarted","Data":"0ca3baaac220fc90e69e3c87c8270494e7a03e1ee1d1db7d78737202dab64916"} Jul 21 18:39:04.213062 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:04.213022 2568 generic.go:356] "Generic (PLEG): container finished" podID="4c03324c-7392-442d-bdd8-013c1debe6af" containerID="12c1b6d19af58729f1312c349275fa5630f7c24b6209db4ea2f31be2a3e6e7ab" exitCode=0 Jul 21 18:39:04.213437 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:04.213068 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf" event={"ID":"4c03324c-7392-442d-bdd8-013c1debe6af","Type":"ContainerDied","Data":"12c1b6d19af58729f1312c349275fa5630f7c24b6209db4ea2f31be2a3e6e7ab"} Jul 21 18:39:05.359479 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:05.359453 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf" Jul 21 18:39:05.512991 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:05.512893 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c03324c-7392-442d-bdd8-013c1debe6af-maas-api-service-ca\") pod \"4c03324c-7392-442d-bdd8-013c1debe6af\" (UID: \"4c03324c-7392-442d-bdd8-013c1debe6af\") " Jul 21 18:39:05.513215 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:05.513192 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c03324c-7392-442d-bdd8-013c1debe6af-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4c03324c-7392-442d-bdd8-013c1debe6af" (UID: "4c03324c-7392-442d-bdd8-013c1debe6af"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:39:05.614487 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:05.614447 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c03324c-7392-442d-bdd8-013c1debe6af-maas-api-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:39:06.225825 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:06.225788 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf" event={"ID":"4c03324c-7392-442d-bdd8-013c1debe6af","Type":"ContainerDied","Data":"0ca3baaac220fc90e69e3c87c8270494e7a03e1ee1d1db7d78737202dab64916"} Jul 21 18:39:06.225825 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:06.225812 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf" Jul 21 18:39:06.226119 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:06.225841 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ca3baaac220fc90e69e3c87c8270494e7a03e1ee1d1db7d78737202dab64916" Jul 21 18:39:11.688603 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:11.688568 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:11.787498 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:11.787464 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:13.367541 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:13.367508 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:13.468095 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:13.468063 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:18.641309 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:18.641277 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5"] Jul 21 18:39:18.641730 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:18.641482 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" podUID="e95eb4ad-526e-4f09-8042-f76b78860e5c" containerName="maas-api" containerID="cri-o://c64c9a41feab619522b0e22c296798f9ed8889a1843b4d6cd7a6438b7a29dd66" gracePeriod=30 Jul 21 18:39:18.658898 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:18.658867 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:18.888409 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:18.888380 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:39:18.921683 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:18.921590 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e95eb4ad-526e-4f09-8042-f76b78860e5c-openshift-service-ca\") pod \"e95eb4ad-526e-4f09-8042-f76b78860e5c\" (UID: \"e95eb4ad-526e-4f09-8042-f76b78860e5c\") " Jul 21 18:39:18.921683 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:18.921633 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e95eb4ad-526e-4f09-8042-f76b78860e5c-maas-api-tls\") pod \"e95eb4ad-526e-4f09-8042-f76b78860e5c\" (UID: \"e95eb4ad-526e-4f09-8042-f76b78860e5c\") " Jul 21 18:39:18.921683 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:18.921653 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5b26r\" (UniqueName: \"kubernetes.io/projected/e95eb4ad-526e-4f09-8042-f76b78860e5c-kube-api-access-5b26r\") pod \"e95eb4ad-526e-4f09-8042-f76b78860e5c\" (UID: \"e95eb4ad-526e-4f09-8042-f76b78860e5c\") " Jul 21 18:39:18.921996 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:18.921971 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e95eb4ad-526e-4f09-8042-f76b78860e5c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e95eb4ad-526e-4f09-8042-f76b78860e5c" (UID: "e95eb4ad-526e-4f09-8042-f76b78860e5c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:39:18.923760 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:18.923730 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e95eb4ad-526e-4f09-8042-f76b78860e5c-kube-api-access-5b26r" (OuterVolumeSpecName: "kube-api-access-5b26r") pod "e95eb4ad-526e-4f09-8042-f76b78860e5c" (UID: "e95eb4ad-526e-4f09-8042-f76b78860e5c"). InnerVolumeSpecName "kube-api-access-5b26r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:39:18.923888 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:18.923784 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e95eb4ad-526e-4f09-8042-f76b78860e5c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e95eb4ad-526e-4f09-8042-f76b78860e5c" (UID: "e95eb4ad-526e-4f09-8042-f76b78860e5c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:39:19.022549 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:19.022512 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e95eb4ad-526e-4f09-8042-f76b78860e5c-openshift-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:39:19.022549 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:19.022542 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e95eb4ad-526e-4f09-8042-f76b78860e5c-maas-api-tls\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:39:19.022549 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:19.022551 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5b26r\" (UniqueName: \"kubernetes.io/projected/e95eb4ad-526e-4f09-8042-f76b78860e5c-kube-api-access-5b26r\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:39:19.300018 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:19.299985 2568 generic.go:356] "Generic (PLEG): container finished" podID="e95eb4ad-526e-4f09-8042-f76b78860e5c" containerID="c64c9a41feab619522b0e22c296798f9ed8889a1843b4d6cd7a6438b7a29dd66" exitCode=0 Jul 21 18:39:19.300197 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:19.300032 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" Jul 21 18:39:19.300197 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:19.300063 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" event={"ID":"e95eb4ad-526e-4f09-8042-f76b78860e5c","Type":"ContainerDied","Data":"c64c9a41feab619522b0e22c296798f9ed8889a1843b4d6cd7a6438b7a29dd66"} Jul 21 18:39:19.300197 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:19.300117 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5" event={"ID":"e95eb4ad-526e-4f09-8042-f76b78860e5c","Type":"ContainerDied","Data":"77d517733ba6d5a39f7543635069b7f6377c7e3d885f6f4529f824da9ce8ce0e"} Jul 21 18:39:19.300197 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:19.300143 2568 scope.go:117] "RemoveContainer" containerID="c64c9a41feab619522b0e22c296798f9ed8889a1843b4d6cd7a6438b7a29dd66" Jul 21 18:39:19.309606 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:19.309588 2568 scope.go:117] "RemoveContainer" containerID="c64c9a41feab619522b0e22c296798f9ed8889a1843b4d6cd7a6438b7a29dd66" Jul 21 18:39:19.309893 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:39:19.309875 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c64c9a41feab619522b0e22c296798f9ed8889a1843b4d6cd7a6438b7a29dd66\": container with ID starting with c64c9a41feab619522b0e22c296798f9ed8889a1843b4d6cd7a6438b7a29dd66 not found: ID does not exist" containerID="c64c9a41feab619522b0e22c296798f9ed8889a1843b4d6cd7a6438b7a29dd66" Jul 21 18:39:19.309968 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:19.309903 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c64c9a41feab619522b0e22c296798f9ed8889a1843b4d6cd7a6438b7a29dd66"} err="failed to get container status \"c64c9a41feab619522b0e22c296798f9ed8889a1843b4d6cd7a6438b7a29dd66\": rpc error: code = NotFound desc = could not find container \"c64c9a41feab619522b0e22c296798f9ed8889a1843b4d6cd7a6438b7a29dd66\": container with ID starting with c64c9a41feab619522b0e22c296798f9ed8889a1843b4d6cd7a6438b7a29dd66 not found: ID does not exist" Jul 21 18:39:19.318591 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:19.318561 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5"] Jul 21 18:39:19.325325 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:19.325299 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9c004f-6d49cf8b4b-9cdp5"] Jul 21 18:39:21.223867 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:21.223830 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e95eb4ad-526e-4f09-8042-f76b78860e5c" path="/var/lib/kubelet/pods/e95eb4ad-526e-4f09-8042-f76b78860e5c/volumes" Jul 21 18:39:21.981234 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:21.981176 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:22.787876 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:22.787842 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf"] Jul 21 18:39:22.791414 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:22.791391 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9c004f-sjczf"] Jul 21 18:39:22.791846 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:22.791824 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:23.207784 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:23.207752 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:23.218417 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:23.218384 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c03324c-7392-442d-bdd8-013c1debe6af" path="/var/lib/kubelet/pods/4c03324c-7392-442d-bdd8-013c1debe6af/volumes" Jul 21 18:39:24.734409 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:24.734374 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:39:29.710204 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:29.710167 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:29.813621 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:29.813594 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:29.994084 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:29.994001 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:30.244030 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.243994 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:30.356194 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.356152 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274"] Jul 21 18:39:30.356501 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.356487 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c03324c-7392-442d-bdd8-013c1debe6af" containerName="revoke-keys" Jul 21 18:39:30.356501 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.356500 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c03324c-7392-442d-bdd8-013c1debe6af" containerName="revoke-keys" Jul 21 18:39:30.356618 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.356526 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e95eb4ad-526e-4f09-8042-f76b78860e5c" containerName="maas-api" Jul 21 18:39:30.356618 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.356532 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e95eb4ad-526e-4f09-8042-f76b78860e5c" containerName="maas-api" Jul 21 18:39:30.356618 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.356587 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c03324c-7392-442d-bdd8-013c1debe6af" containerName="revoke-keys" Jul 21 18:39:30.356618 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.356597 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e95eb4ad-526e-4f09-8042-f76b78860e5c" containerName="maas-api" Jul 21 18:39:30.361442 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.361419 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274" Jul 21 18:39:30.364162 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.364136 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:39:30.368914 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.368889 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274"] Jul 21 18:39:30.425927 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.425889 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ab5cda5-4e84-417b-8fc1-33b2cff75ae6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-057ec0-hv274\" (UID: \"4ab5cda5-4e84-417b-8fc1-33b2cff75ae6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274" Jul 21 18:39:30.526909 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.526866 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ab5cda5-4e84-417b-8fc1-33b2cff75ae6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-057ec0-hv274\" (UID: \"4ab5cda5-4e84-417b-8fc1-33b2cff75ae6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274" Jul 21 18:39:30.527672 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.527646 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ab5cda5-4e84-417b-8fc1-33b2cff75ae6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-057ec0-hv274\" (UID: \"4ab5cda5-4e84-417b-8fc1-33b2cff75ae6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274" Jul 21 18:39:30.674747 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.674655 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274" Jul 21 18:39:30.818224 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:30.818192 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274"] Jul 21 18:39:30.820134 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:39:30.820107 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab5cda5_4e84_417b_8fc1_33b2cff75ae6.slice/crio-49de7ba5a966eaa5869ec9f7863aed7eb4b2ceabff830a8672b86e40b51e8bc6 WatchSource:0}: Error finding container 49de7ba5a966eaa5869ec9f7863aed7eb4b2ceabff830a8672b86e40b51e8bc6: Status 404 returned error can't find the container with id 49de7ba5a966eaa5869ec9f7863aed7eb4b2ceabff830a8672b86e40b51e8bc6 Jul 21 18:39:31.022686 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:31.022600 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:31.123992 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:31.123930 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:31.365122 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:31.365090 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274" event={"ID":"4ab5cda5-4e84-417b-8fc1-33b2cff75ae6","Type":"ContainerStarted","Data":"bc822c7ffdeb609944898c887e8081fd0215ee0e8b66f4423418e70e6d7ba6f4"} Jul 21 18:39:31.365288 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:31.365153 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274" event={"ID":"4ab5cda5-4e84-417b-8fc1-33b2cff75ae6","Type":"ContainerStarted","Data":"49de7ba5a966eaa5869ec9f7863aed7eb4b2ceabff830a8672b86e40b51e8bc6"} Jul 21 18:39:31.387056 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:31.387000 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274" podStartSLOduration=1.386982996 podStartE2EDuration="1.386982996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:39:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:39:31.384706417 +0000 UTC m=+2372.653017858" watchObservedRunningTime="2026-07-21 18:39:31.386982996 +0000 UTC m=+2372.655294415" Jul 21 18:39:32.371797 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:32.371754 2568 generic.go:356] "Generic (PLEG): container finished" podID="4ab5cda5-4e84-417b-8fc1-33b2cff75ae6" containerID="bc822c7ffdeb609944898c887e8081fd0215ee0e8b66f4423418e70e6d7ba6f4" exitCode=0 Jul 21 18:39:32.372213 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:32.371848 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274" event={"ID":"4ab5cda5-4e84-417b-8fc1-33b2cff75ae6","Type":"ContainerDied","Data":"bc822c7ffdeb609944898c887e8081fd0215ee0e8b66f4423418e70e6d7ba6f4"} Jul 21 18:39:33.510109 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:33.510083 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274" Jul 21 18:39:33.653820 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:33.653708 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ab5cda5-4e84-417b-8fc1-33b2cff75ae6-maas-api-service-ca\") pod \"4ab5cda5-4e84-417b-8fc1-33b2cff75ae6\" (UID: \"4ab5cda5-4e84-417b-8fc1-33b2cff75ae6\") " Jul 21 18:39:33.654186 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:33.654151 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ab5cda5-4e84-417b-8fc1-33b2cff75ae6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4ab5cda5-4e84-417b-8fc1-33b2cff75ae6" (UID: "4ab5cda5-4e84-417b-8fc1-33b2cff75ae6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:39:33.755028 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:33.754977 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ab5cda5-4e84-417b-8fc1-33b2cff75ae6-maas-api-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:39:34.383700 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:34.383669 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274" Jul 21 18:39:34.383899 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:34.383714 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274" event={"ID":"4ab5cda5-4e84-417b-8fc1-33b2cff75ae6","Type":"ContainerDied","Data":"49de7ba5a966eaa5869ec9f7863aed7eb4b2ceabff830a8672b86e40b51e8bc6"} Jul 21 18:39:34.383899 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:34.383782 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49de7ba5a966eaa5869ec9f7863aed7eb4b2ceabff830a8672b86e40b51e8bc6" Jul 21 18:39:46.125386 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.125352 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz"] Jul 21 18:39:46.125860 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.125581 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" podUID="13b67586-2374-4bb1-82aa-e6aab1870048" containerName="maas-api" containerID="cri-o://0180066acde8bb08868d4837f36e6eee40f090674afb3dd4c2ffcc5d80384fd9" gracePeriod=30 Jul 21 18:39:46.159182 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.159142 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:46.370140 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.370113 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:39:46.447748 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.447649 2568 generic.go:356] "Generic (PLEG): container finished" podID="13b67586-2374-4bb1-82aa-e6aab1870048" containerID="0180066acde8bb08868d4837f36e6eee40f090674afb3dd4c2ffcc5d80384fd9" exitCode=0 Jul 21 18:39:46.447748 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.447713 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" Jul 21 18:39:46.447964 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.447745 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" event={"ID":"13b67586-2374-4bb1-82aa-e6aab1870048","Type":"ContainerDied","Data":"0180066acde8bb08868d4837f36e6eee40f090674afb3dd4c2ffcc5d80384fd9"} Jul 21 18:39:46.447964 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.447822 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz" event={"ID":"13b67586-2374-4bb1-82aa-e6aab1870048","Type":"ContainerDied","Data":"720d166cae3599aa184ac19265fd3c63f77a45411d754e0cb9eca69cab37db5f"} Jul 21 18:39:46.447964 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.447844 2568 scope.go:117] "RemoveContainer" containerID="0180066acde8bb08868d4837f36e6eee40f090674afb3dd4c2ffcc5d80384fd9" Jul 21 18:39:46.456694 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.456674 2568 scope.go:117] "RemoveContainer" containerID="0180066acde8bb08868d4837f36e6eee40f090674afb3dd4c2ffcc5d80384fd9" Jul 21 18:39:46.456983 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:39:46.456963 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0180066acde8bb08868d4837f36e6eee40f090674afb3dd4c2ffcc5d80384fd9\": container with ID starting with 0180066acde8bb08868d4837f36e6eee40f090674afb3dd4c2ffcc5d80384fd9 not found: ID does not exist" containerID="0180066acde8bb08868d4837f36e6eee40f090674afb3dd4c2ffcc5d80384fd9" Jul 21 18:39:46.457043 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.456994 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0180066acde8bb08868d4837f36e6eee40f090674afb3dd4c2ffcc5d80384fd9"} err="failed to get container status \"0180066acde8bb08868d4837f36e6eee40f090674afb3dd4c2ffcc5d80384fd9\": rpc error: code = NotFound desc = could not find container \"0180066acde8bb08868d4837f36e6eee40f090674afb3dd4c2ffcc5d80384fd9\": container with ID starting with 0180066acde8bb08868d4837f36e6eee40f090674afb3dd4c2ffcc5d80384fd9 not found: ID does not exist" Jul 21 18:39:46.461244 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.461226 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b67586-2374-4bb1-82aa-e6aab1870048-openshift-service-ca\") pod \"13b67586-2374-4bb1-82aa-e6aab1870048\" (UID: \"13b67586-2374-4bb1-82aa-e6aab1870048\") " Jul 21 18:39:46.461306 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.461260 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b67586-2374-4bb1-82aa-e6aab1870048-maas-api-tls\") pod \"13b67586-2374-4bb1-82aa-e6aab1870048\" (UID: \"13b67586-2374-4bb1-82aa-e6aab1870048\") " Jul 21 18:39:46.461359 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.461312 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6xmj\" (UniqueName: \"kubernetes.io/projected/13b67586-2374-4bb1-82aa-e6aab1870048-kube-api-access-x6xmj\") pod \"13b67586-2374-4bb1-82aa-e6aab1870048\" (UID: \"13b67586-2374-4bb1-82aa-e6aab1870048\") " Jul 21 18:39:46.461599 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.461576 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13b67586-2374-4bb1-82aa-e6aab1870048-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "13b67586-2374-4bb1-82aa-e6aab1870048" (UID: "13b67586-2374-4bb1-82aa-e6aab1870048"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:39:46.463282 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.463263 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13b67586-2374-4bb1-82aa-e6aab1870048-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "13b67586-2374-4bb1-82aa-e6aab1870048" (UID: "13b67586-2374-4bb1-82aa-e6aab1870048"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:39:46.463346 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.463272 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13b67586-2374-4bb1-82aa-e6aab1870048-kube-api-access-x6xmj" (OuterVolumeSpecName: "kube-api-access-x6xmj") pod "13b67586-2374-4bb1-82aa-e6aab1870048" (UID: "13b67586-2374-4bb1-82aa-e6aab1870048"). InnerVolumeSpecName "kube-api-access-x6xmj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:39:46.562999 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.562922 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b67586-2374-4bb1-82aa-e6aab1870048-maas-api-tls\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:39:46.562999 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.562998 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6xmj\" (UniqueName: \"kubernetes.io/projected/13b67586-2374-4bb1-82aa-e6aab1870048-kube-api-access-x6xmj\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:39:46.563266 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.563017 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b67586-2374-4bb1-82aa-e6aab1870048-openshift-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:39:46.773999 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.773677 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz"] Jul 21 18:39:46.775902 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:46.775877 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-057ec0-89c4f75f5-46hcz"] Jul 21 18:39:47.218104 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:47.218069 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13b67586-2374-4bb1-82aa-e6aab1870048" path="/var/lib/kubelet/pods/13b67586-2374-4bb1-82aa-e6aab1870048/volumes" Jul 21 18:39:50.528316 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:50.528279 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274"] Jul 21 18:39:50.533824 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:50.533790 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-057ec0-hv274"] Jul 21 18:39:50.565142 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:50.565103 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:50.980185 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:50.980150 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:51.075462 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:51.075425 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:39:51.218921 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:51.218885 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ab5cda5-4e84-417b-8fc1-33b2cff75ae6" path="/var/lib/kubelet/pods/4ab5cda5-4e84-417b-8fc1-33b2cff75ae6/volumes" Jul 21 18:39:57.890073 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:57.890039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:58.004259 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:58.004231 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:58.196375 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:58.196286 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:58.415069 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:58.415034 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:58.797777 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:58.797744 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:59.245897 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:59.245854 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:59.351011 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:59.350979 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:39:59.354559 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:59.354532 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:39:59.364326 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:59.364290 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:39:59.367838 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:59.367818 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:39:59.412651 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:59.412614 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:39:59.992131 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:39:59.992091 2568 scope.go:117] "RemoveContainer" containerID="27b5306cb328cd408b537bdf597f3b94227111a0d282c3b025bb08d1fdaeb34f" Jul 21 18:40:01.719956 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:01.719894 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:01.837400 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:01.837367 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:02.180919 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.180884 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g"] Jul 21 18:40:02.181233 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.181221 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b67586-2374-4bb1-82aa-e6aab1870048" containerName="maas-api" Jul 21 18:40:02.181284 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.181235 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b67586-2374-4bb1-82aa-e6aab1870048" containerName="maas-api" Jul 21 18:40:02.181284 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.181252 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ab5cda5-4e84-417b-8fc1-33b2cff75ae6" containerName="revoke-keys" Jul 21 18:40:02.181284 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.181257 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ab5cda5-4e84-417b-8fc1-33b2cff75ae6" containerName="revoke-keys" Jul 21 18:40:02.181391 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.181308 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b67586-2374-4bb1-82aa-e6aab1870048" containerName="maas-api" Jul 21 18:40:02.181391 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.181319 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ab5cda5-4e84-417b-8fc1-33b2cff75ae6" containerName="revoke-keys" Jul 21 18:40:02.184306 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.184288 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:40:02.193702 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.193684 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-c1a1f4-serving-cert\"" Jul 21 18:40:02.193793 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.193724 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 18:40:02.193793 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.193725 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 18:40:02.194762 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.194747 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sh2jm\"" Jul 21 18:40:02.202387 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.202360 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g"] Jul 21 18:40:02.244205 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.244167 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:02.303354 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.303313 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97926db-d3fd-4560-93e4-2f3b346cb374-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g\" (UID: \"a97926db-d3fd-4560-93e4-2f3b346cb374\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:40:02.303526 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.303386 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxh67\" (UniqueName: \"kubernetes.io/projected/a97926db-d3fd-4560-93e4-2f3b346cb374-kube-api-access-hxh67\") pod \"maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g\" (UID: \"a97926db-d3fd-4560-93e4-2f3b346cb374\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:40:02.303526 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.303421 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a97926db-d3fd-4560-93e4-2f3b346cb374-maas-api-tls\") pod \"maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g\" (UID: \"a97926db-d3fd-4560-93e4-2f3b346cb374\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:40:02.402974 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.402895 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:02.404461 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.404440 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97926db-d3fd-4560-93e4-2f3b346cb374-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g\" (UID: \"a97926db-d3fd-4560-93e4-2f3b346cb374\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:40:02.404550 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.404498 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxh67\" (UniqueName: \"kubernetes.io/projected/a97926db-d3fd-4560-93e4-2f3b346cb374-kube-api-access-hxh67\") pod \"maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g\" (UID: \"a97926db-d3fd-4560-93e4-2f3b346cb374\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:40:02.404550 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.404523 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a97926db-d3fd-4560-93e4-2f3b346cb374-maas-api-tls\") pod \"maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g\" (UID: \"a97926db-d3fd-4560-93e4-2f3b346cb374\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:40:02.405080 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.405063 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97926db-d3fd-4560-93e4-2f3b346cb374-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g\" (UID: \"a97926db-d3fd-4560-93e4-2f3b346cb374\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:40:02.406870 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.406850 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a97926db-d3fd-4560-93e4-2f3b346cb374-maas-api-tls\") pod \"maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g\" (UID: \"a97926db-d3fd-4560-93e4-2f3b346cb374\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:40:02.417346 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.417325 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxh67\" (UniqueName: \"kubernetes.io/projected/a97926db-d3fd-4560-93e4-2f3b346cb374-kube-api-access-hxh67\") pod \"maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g\" (UID: \"a97926db-d3fd-4560-93e4-2f3b346cb374\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:40:02.493994 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.493884 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:40:02.642294 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.642202 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g"] Jul 21 18:40:02.645156 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:40:02.645128 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda97926db_d3fd_4560_93e4_2f3b346cb374.slice/crio-4b5a6ddb45f9e66dece1a0484334495a16c7412d8abb6034171b763ef534e52f WatchSource:0}: Error finding container 4b5a6ddb45f9e66dece1a0484334495a16c7412d8abb6034171b763ef534e52f: Status 404 returned error can't find the container with id 4b5a6ddb45f9e66dece1a0484334495a16c7412d8abb6034171b763ef534e52f Jul 21 18:40:02.783774 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:02.783675 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:03.158189 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:03.158114 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:03.311266 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:03.311237 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:03.538192 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:03.538084 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" event={"ID":"a97926db-d3fd-4560-93e4-2f3b346cb374","Type":"ContainerStarted","Data":"073e22cc95a02c28d18f664c5707cedd434ed72585bcff83d30f1fa498c9172e"} Jul 21 18:40:03.538192 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:03.538148 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" event={"ID":"a97926db-d3fd-4560-93e4-2f3b346cb374","Type":"ContainerStarted","Data":"4b5a6ddb45f9e66dece1a0484334495a16c7412d8abb6034171b763ef534e52f"} Jul 21 18:40:03.538192 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:03.538178 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:40:03.567800 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:03.567749 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" podStartSLOduration=0.994419444 podStartE2EDuration="1.56773168s" totalImagesPullingTime="573.312236ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:40:03.566569631 +0000 UTC m=+2404.834881074" watchObservedRunningTime="2026-07-21 18:40:03.56773168 +0000 UTC m=+2404.836043120" Jul 21 18:40:05.240375 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:05.240332 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:40:06.544501 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:06.544467 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:06.892104 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:06.892066 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:07.215586 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:07.215500 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:07.471245 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:07.471156 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:09.549081 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:09.549052 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:40:10.250207 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.250172 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm"] Jul 21 18:40:10.253730 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.253712 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:40:10.256188 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.256170 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-0be774-serving-cert\"" Jul 21 18:40:10.264896 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.264861 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm"] Jul 21 18:40:10.267386 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.267358 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca764bb0-54dc-4855-a53a-0f55799886bc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0be774-646487775f-6qnmm\" (UID: \"ca764bb0-54dc-4855-a53a-0f55799886bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:40:10.267502 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.267411 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca764bb0-54dc-4855-a53a-0f55799886bc-maas-api-tls\") pod \"maas-api-e2e-shared-b-0be774-646487775f-6qnmm\" (UID: \"ca764bb0-54dc-4855-a53a-0f55799886bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:40:10.267568 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.267491 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsrqs\" (UniqueName: \"kubernetes.io/projected/ca764bb0-54dc-4855-a53a-0f55799886bc-kube-api-access-wsrqs\") pod \"maas-api-e2e-shared-b-0be774-646487775f-6qnmm\" (UID: \"ca764bb0-54dc-4855-a53a-0f55799886bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:40:10.368295 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.368262 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca764bb0-54dc-4855-a53a-0f55799886bc-maas-api-tls\") pod \"maas-api-e2e-shared-b-0be774-646487775f-6qnmm\" (UID: \"ca764bb0-54dc-4855-a53a-0f55799886bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:40:10.368467 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.368311 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsrqs\" (UniqueName: \"kubernetes.io/projected/ca764bb0-54dc-4855-a53a-0f55799886bc-kube-api-access-wsrqs\") pod \"maas-api-e2e-shared-b-0be774-646487775f-6qnmm\" (UID: \"ca764bb0-54dc-4855-a53a-0f55799886bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:40:10.368467 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.368347 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca764bb0-54dc-4855-a53a-0f55799886bc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0be774-646487775f-6qnmm\" (UID: \"ca764bb0-54dc-4855-a53a-0f55799886bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:40:10.368467 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:40:10.368426 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-serving-cert: secret "maas-api-e2e-shared-b-0be774-serving-cert" not found Jul 21 18:40:10.368575 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:40:10.368503 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca764bb0-54dc-4855-a53a-0f55799886bc-maas-api-tls podName:ca764bb0-54dc-4855-a53a-0f55799886bc nodeName:}" failed. No retries permitted until 2026-07-21 18:40:10.868482329 +0000 UTC m=+2412.136793761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca764bb0-54dc-4855-a53a-0f55799886bc-maas-api-tls") pod "maas-api-e2e-shared-b-0be774-646487775f-6qnmm" (UID: "ca764bb0-54dc-4855-a53a-0f55799886bc") : secret "maas-api-e2e-shared-b-0be774-serving-cert" not found Jul 21 18:40:10.368960 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.368922 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca764bb0-54dc-4855-a53a-0f55799886bc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0be774-646487775f-6qnmm\" (UID: \"ca764bb0-54dc-4855-a53a-0f55799886bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:40:10.389802 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.389765 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsrqs\" (UniqueName: \"kubernetes.io/projected/ca764bb0-54dc-4855-a53a-0f55799886bc-kube-api-access-wsrqs\") pod \"maas-api-e2e-shared-b-0be774-646487775f-6qnmm\" (UID: \"ca764bb0-54dc-4855-a53a-0f55799886bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:40:10.418420 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.418383 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:10.556771 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.556674 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:10.740584 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.740551 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:10.873517 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.873478 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca764bb0-54dc-4855-a53a-0f55799886bc-maas-api-tls\") pod \"maas-api-e2e-shared-b-0be774-646487775f-6qnmm\" (UID: \"ca764bb0-54dc-4855-a53a-0f55799886bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:40:10.876150 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:10.876118 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca764bb0-54dc-4855-a53a-0f55799886bc-maas-api-tls\") pod \"maas-api-e2e-shared-b-0be774-646487775f-6qnmm\" (UID: \"ca764bb0-54dc-4855-a53a-0f55799886bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:40:11.008579 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:11.008548 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:11.111679 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:11.111623 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:11.165731 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:11.165641 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:40:11.306093 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:11.306067 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm"] Jul 21 18:40:11.308316 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:40:11.308289 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca764bb0_54dc_4855_a53a_0f55799886bc.slice/crio-a010b5a27b0f9399078dea37dc71c9e65b743b921bec8a8f4365bfbd5bc4a68f WatchSource:0}: Error finding container a010b5a27b0f9399078dea37dc71c9e65b743b921bec8a8f4365bfbd5bc4a68f: Status 404 returned error can't find the container with id a010b5a27b0f9399078dea37dc71c9e65b743b921bec8a8f4365bfbd5bc4a68f Jul 21 18:40:11.583195 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:11.583110 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" event={"ID":"ca764bb0-54dc-4855-a53a-0f55799886bc","Type":"ContainerStarted","Data":"a010b5a27b0f9399078dea37dc71c9e65b743b921bec8a8f4365bfbd5bc4a68f"} Jul 21 18:40:12.590278 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:12.590238 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" event={"ID":"ca764bb0-54dc-4855-a53a-0f55799886bc","Type":"ContainerStarted","Data":"3e5e94283adf08c7e6f043114138d25aa6c9af6d3aa3a612bf846163da32bc40"} Jul 21 18:40:12.590687 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:12.590344 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:40:12.614464 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:12.614411 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" podStartSLOduration=2.169293428 podStartE2EDuration="2.614396745s" totalImagesPullingTime="445.103317ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:40:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:40:12.613045065 +0000 UTC m=+2413.881356506" watchObservedRunningTime="2026-07-21 18:40:12.614396745 +0000 UTC m=+2413.882708186" Jul 21 18:40:14.484321 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:14.484285 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:18.601945 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:18.601893 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:40:19.570582 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:19.570550 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:40:24.080718 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:24.080633 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:24.181781 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:24.181749 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:31.925415 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:31.925379 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:32.024601 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:32.024564 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:35.911117 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:35.911080 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:40:37.846278 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:40:37.846239 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:41:05.668010 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:41:05.667969 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:41:08.841662 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:41:08.841622 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:41:22.036260 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:41:22.036220 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:41:36.026916 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:41:36.026877 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:41:40.749354 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:41:40.749319 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:41:41.181111 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:41:41.181064 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:41:55.247276 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:41:55.247187 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:42:12.541814 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:42:12.541779 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:42:26.633433 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:42:26.633387 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:42:42.131257 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:42:42.131219 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:42:56.107369 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:42:56.107323 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:42:57.083830 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:42:57.083783 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:43:00.828655 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:43:00.828611 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:43:05.165413 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:43:05.165358 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:43:13.230890 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:43:13.230852 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:43:40.563594 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:43:40.563510 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:43:44.526711 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:43:44.526672 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:44:00.044076 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:44:00.044042 2568 scope.go:117] "RemoveContainer" containerID="ba7878cf7d5d4bc9b87af084aa837befb5c8cb6504c1f69e1b51b86921d1f367" Jul 21 18:44:03.939128 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:44:03.939091 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:44:24.442112 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:44:24.442073 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:44:29.679821 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:44:29.679788 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:44:30.494449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:44:30.494416 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:44:33.024680 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:44:33.024639 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:44:41.316847 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:44:41.316814 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:44:42.330036 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:44:42.329967 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:44:51.641365 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:44:51.641273 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:44:59.393924 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:44:59.393897 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:44:59.399107 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:44:59.399084 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:44:59.400541 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:44:59.400522 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:44:59.405265 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:44:59.405246 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7h7m4_6746d28c-eb2a-40ec-ab6d-da2e086116dc/ovn-acl-logging/0.log" Jul 21 18:45:00.068249 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.068214 2568 scope.go:117] "RemoveContainer" containerID="be885deab41681001cd7c7e00f4104ffe97bd4a3050f623265f48b3db1ba30a6" Jul 21 18:45:00.139604 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.139563 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0be774-29744325-bg8fs"] Jul 21 18:45:00.144694 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.144668 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0be774-29744325-bg8fs" Jul 21 18:45:00.146179 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.146161 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744325-kkhxd"] Jul 21 18:45:00.147147 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.147127 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:45:00.149353 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.149322 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c1a1f4-29744325-bcwwf"] Jul 21 18:45:00.149497 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.149474 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744325-kkhxd" Jul 21 18:45:00.154731 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.152998 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0be774-29744325-bg8fs"] Jul 21 18:45:00.154731 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.153108 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c1a1f4-29744325-bcwwf" Jul 21 18:45:00.159525 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.159499 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0be774-29744325-bg8fs" Jul 21 18:45:00.159993 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.159580 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744325-kkhxd"] Jul 21 18:45:00.163318 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.163297 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c1a1f4-29744325-bcwwf"] Jul 21 18:45:00.169229 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.169210 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744325-kkhxd" Jul 21 18:45:00.207294 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.207267 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c1a1f4-29744325-bcwwf" Jul 21 18:45:00.324968 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.324840 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0be774-29744325-bg8fs"] Jul 21 18:45:00.334576 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:45:00.332055 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff353bf9_4ff4_4af0_92d6_ebe5eb8a782b.slice/crio-69b7344a5cffa4a02a7514566748a836670b1b98db09c0f282951695fb8b5ffd WatchSource:0}: Error finding container 69b7344a5cffa4a02a7514566748a836670b1b98db09c0f282951695fb8b5ffd: Status 404 returned error can't find the container with id 69b7344a5cffa4a02a7514566748a836670b1b98db09c0f282951695fb8b5ffd Jul 21 18:45:00.351742 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.351708 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744325-kkhxd"] Jul 21 18:45:00.353032 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:45:00.353002 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1a1b537_404d_4c3a_9229_a03f1191eb29.slice/crio-d544d4688d0a91cb8f660481a917c894ac0f60bb0054928e30e81cfc5d4542ce WatchSource:0}: Error finding container d544d4688d0a91cb8f660481a917c894ac0f60bb0054928e30e81cfc5d4542ce: Status 404 returned error can't find the container with id d544d4688d0a91cb8f660481a917c894ac0f60bb0054928e30e81cfc5d4542ce Jul 21 18:45:00.377188 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:00.377167 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c1a1f4-29744325-bcwwf"] Jul 21 18:45:00.383030 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:45:00.383001 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e919a63_fb1b_4de6_a768_c0fbbe0267ca.slice/crio-1d64cd2e4a61dd97d7df993b96c5f11cd68b2fffde2b1a4a140b0da44744ef57 WatchSource:0}: Error finding container 1d64cd2e4a61dd97d7df993b96c5f11cd68b2fffde2b1a4a140b0da44744ef57: Status 404 returned error can't find the container with id 1d64cd2e4a61dd97d7df993b96c5f11cd68b2fffde2b1a4a140b0da44744ef57 Jul 21 18:45:01.129491 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:01.129451 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0be774-29744325-bg8fs" event={"ID":"ff353bf9-4ff4-4af0-92d6-ebe5eb8a782b","Type":"ContainerStarted","Data":"7abe9939147b2f330ede70ac43a33a2da3b8029d226ad8d1f0a8e3f8776cce31"} Jul 21 18:45:01.129899 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:01.129532 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0be774-29744325-bg8fs" event={"ID":"ff353bf9-4ff4-4af0-92d6-ebe5eb8a782b","Type":"ContainerStarted","Data":"69b7344a5cffa4a02a7514566748a836670b1b98db09c0f282951695fb8b5ffd"} Jul 21 18:45:01.130984 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:01.130958 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c1a1f4-29744325-bcwwf" event={"ID":"7e919a63-fb1b-4de6-a768-c0fbbe0267ca","Type":"ContainerStarted","Data":"79b2ce7deea6b4329da47d701f5e24c90142b133772bbdba22bac39c00b763e4"} Jul 21 18:45:01.131080 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:01.131003 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c1a1f4-29744325-bcwwf" event={"ID":"7e919a63-fb1b-4de6-a768-c0fbbe0267ca","Type":"ContainerStarted","Data":"1d64cd2e4a61dd97d7df993b96c5f11cd68b2fffde2b1a4a140b0da44744ef57"} Jul 21 18:45:01.132292 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:01.132267 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744325-kkhxd" event={"ID":"c1a1b537-404d-4c3a-9229-a03f1191eb29","Type":"ContainerStarted","Data":"4ceb73af7f3c3e28e2cca0e722d04960b1c4cf12d4575d6f578fa9669d3b370f"} Jul 21 18:45:01.132381 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:01.132305 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744325-kkhxd" event={"ID":"c1a1b537-404d-4c3a-9229-a03f1191eb29","Type":"ContainerStarted","Data":"d544d4688d0a91cb8f660481a917c894ac0f60bb0054928e30e81cfc5d4542ce"} Jul 21 18:45:01.145623 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:01.145568 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0be774-29744325-bg8fs" podStartSLOduration=1.145555428 podStartE2EDuration="1.145555428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:45:01.143157659 +0000 UTC m=+2702.411469101" watchObservedRunningTime="2026-07-21 18:45:01.145555428 +0000 UTC m=+2702.413866867" Jul 21 18:45:01.158744 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:01.158694 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744325-kkhxd" podStartSLOduration=1.158680534 podStartE2EDuration="1.158680534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:45:01.156521425 +0000 UTC m=+2702.424832880" watchObservedRunningTime="2026-07-21 18:45:01.158680534 +0000 UTC m=+2702.426991975" Jul 21 18:45:01.170404 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:01.170359 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c1a1f4-29744325-bcwwf" podStartSLOduration=1.170342674 podStartE2EDuration="1.170342674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:45:01.169897576 +0000 UTC m=+2702.438209015" watchObservedRunningTime="2026-07-21 18:45:01.170342674 +0000 UTC m=+2702.438654114" Jul 21 18:45:02.139960 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:02.139901 2568 generic.go:356] "Generic (PLEG): container finished" podID="ff353bf9-4ff4-4af0-92d6-ebe5eb8a782b" containerID="7abe9939147b2f330ede70ac43a33a2da3b8029d226ad8d1f0a8e3f8776cce31" exitCode=0 Jul 21 18:45:02.140373 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:02.139968 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0be774-29744325-bg8fs" event={"ID":"ff353bf9-4ff4-4af0-92d6-ebe5eb8a782b","Type":"ContainerDied","Data":"7abe9939147b2f330ede70ac43a33a2da3b8029d226ad8d1f0a8e3f8776cce31"} Jul 21 18:45:02.141536 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:02.141515 2568 generic.go:356] "Generic (PLEG): container finished" podID="7e919a63-fb1b-4de6-a768-c0fbbe0267ca" containerID="79b2ce7deea6b4329da47d701f5e24c90142b133772bbdba22bac39c00b763e4" exitCode=0 Jul 21 18:45:02.141649 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:02.141588 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c1a1f4-29744325-bcwwf" event={"ID":"7e919a63-fb1b-4de6-a768-c0fbbe0267ca","Type":"ContainerDied","Data":"79b2ce7deea6b4329da47d701f5e24c90142b133772bbdba22bac39c00b763e4"} Jul 21 18:45:02.142866 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:02.142845 2568 generic.go:356] "Generic (PLEG): container finished" podID="c1a1b537-404d-4c3a-9229-a03f1191eb29" containerID="4ceb73af7f3c3e28e2cca0e722d04960b1c4cf12d4575d6f578fa9669d3b370f" exitCode=0 Jul 21 18:45:02.142983 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:02.142904 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744325-kkhxd" event={"ID":"c1a1b537-404d-4c3a-9229-a03f1191eb29","Type":"ContainerDied","Data":"4ceb73af7f3c3e28e2cca0e722d04960b1c4cf12d4575d6f578fa9669d3b370f"} Jul 21 18:45:02.569374 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:02.569296 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:45:02.665888 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:02.665843 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:45:03.297383 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:03.297350 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0be774-29744325-bg8fs" Jul 21 18:45:03.347801 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:03.347775 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744325-kkhxd" Jul 21 18:45:03.351289 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:03.351266 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c1a1f4-29744325-bcwwf" Jul 21 18:45:04.155303 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:04.155273 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744325-kkhxd" Jul 21 18:45:04.155478 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:04.155369 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744325-kkhxd" event={"ID":"c1a1b537-404d-4c3a-9229-a03f1191eb29","Type":"ContainerDied","Data":"d544d4688d0a91cb8f660481a917c894ac0f60bb0054928e30e81cfc5d4542ce"} Jul 21 18:45:04.155478 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:04.155408 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d544d4688d0a91cb8f660481a917c894ac0f60bb0054928e30e81cfc5d4542ce" Jul 21 18:45:04.156619 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:04.156594 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0be774-29744325-bg8fs" event={"ID":"ff353bf9-4ff4-4af0-92d6-ebe5eb8a782b","Type":"ContainerDied","Data":"69b7344a5cffa4a02a7514566748a836670b1b98db09c0f282951695fb8b5ffd"} Jul 21 18:45:04.156762 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:04.156636 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69b7344a5cffa4a02a7514566748a836670b1b98db09c0f282951695fb8b5ffd" Jul 21 18:45:04.156762 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:04.156604 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0be774-29744325-bg8fs" Jul 21 18:45:04.157861 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:04.157839 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c1a1f4-29744325-bcwwf" event={"ID":"7e919a63-fb1b-4de6-a768-c0fbbe0267ca","Type":"ContainerDied","Data":"1d64cd2e4a61dd97d7df993b96c5f11cd68b2fffde2b1a4a140b0da44744ef57"} Jul 21 18:45:04.157979 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:04.157870 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c1a1f4-29744325-bcwwf" Jul 21 18:45:04.157979 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:04.157878 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d64cd2e4a61dd97d7df993b96c5f11cd68b2fffde2b1a4a140b0da44744ef57" Jul 21 18:45:12.332467 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:12.332417 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:45:12.733413 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:12.733375 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:45:16.631909 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:16.631869 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:45:24.890990 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:24.890953 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:45:25.236094 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:25.236014 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:45:31.588114 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:31.588079 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:45:31.690053 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:31.690018 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:45:32.632766 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:32.632733 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:45:32.695983 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:32.695929 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:45:32.790388 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:32.790338 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:45:47.726361 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:47.726323 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:45:55.187224 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:55.187173 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:45:56.160262 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:56.160228 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:45:57.048607 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.048560 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:45:57.102710 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.102675 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7562-140036458843696"] Jul 21 18:45:57.103083 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.103068 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e919a63-fb1b-4de6-a768-c0fbbe0267ca" containerName="cleanup" Jul 21 18:45:57.103131 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.103084 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e919a63-fb1b-4de6-a768-c0fbbe0267ca" containerName="cleanup" Jul 21 18:45:57.103131 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.103096 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff353bf9-4ff4-4af0-92d6-ebe5eb8a782b" containerName="cleanup" Jul 21 18:45:57.103131 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.103101 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff353bf9-4ff4-4af0-92d6-ebe5eb8a782b" containerName="cleanup" Jul 21 18:45:57.103131 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.103119 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1a1b537-404d-4c3a-9229-a03f1191eb29" containerName="cleanup" Jul 21 18:45:57.103131 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.103127 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1a1b537-404d-4c3a-9229-a03f1191eb29" containerName="cleanup" Jul 21 18:45:57.103283 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.103183 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff353bf9-4ff4-4af0-92d6-ebe5eb8a782b" containerName="cleanup" Jul 21 18:45:57.103283 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.103191 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1a1b537-404d-4c3a-9229-a03f1191eb29" containerName="cleanup" Jul 21 18:45:57.103283 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.103202 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e919a63-fb1b-4de6-a768-c0fbbe0267ca" containerName="cleanup" Jul 21 18:45:57.106343 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.106326 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458843696" Jul 21 18:45:57.109269 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.109167 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lfsn8\"" Jul 21 18:45:57.111156 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.111134 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7562-140036458843696"] Jul 21 18:45:57.246593 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.246552 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwqnb\" (UniqueName: \"kubernetes.io/projected/4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1-kube-api-access-vwqnb\") pod \"test-curl-7562-140036458843696\" (UID: \"4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1\") " pod="opendatahub/test-curl-7562-140036458843696" Jul 21 18:45:57.347684 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.347650 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwqnb\" (UniqueName: \"kubernetes.io/projected/4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1-kube-api-access-vwqnb\") pod \"test-curl-7562-140036458843696\" (UID: \"4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1\") " pod="opendatahub/test-curl-7562-140036458843696" Jul 21 18:45:57.356182 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.356150 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwqnb\" (UniqueName: \"kubernetes.io/projected/4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1-kube-api-access-vwqnb\") pod \"test-curl-7562-140036458843696\" (UID: \"4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1\") " pod="opendatahub/test-curl-7562-140036458843696" Jul 21 18:45:57.417008 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.416963 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458843696" Jul 21 18:45:57.548279 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.548252 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7562-140036458843696"] Jul 21 18:45:57.550233 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:45:57.550204 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ba518d5_5930_4ee4_a3fe_b7c65cb19ab1.slice/crio-d252f23fd13373ac50c6da5496f8590e58c70be432b2cb20d1d09356ff4ef68e WatchSource:0}: Error finding container d252f23fd13373ac50c6da5496f8590e58c70be432b2cb20d1d09356ff4ef68e: Status 404 returned error can't find the container with id d252f23fd13373ac50c6da5496f8590e58c70be432b2cb20d1d09356ff4ef68e Jul 21 18:45:57.551476 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:57.551456 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 18:45:58.448858 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:58.448816 2568 generic.go:356] "Generic (PLEG): container finished" podID="4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1" containerID="feb2b8285bdb6b9e97cc568da5579639a331f4da6a8d5f6edab29595c7a52483" exitCode=0 Jul 21 18:45:58.449292 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:58.448870 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7562-140036458843696" event={"ID":"4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1","Type":"ContainerDied","Data":"feb2b8285bdb6b9e97cc568da5579639a331f4da6a8d5f6edab29595c7a52483"} Jul 21 18:45:58.449292 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:58.448897 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7562-140036458843696" event={"ID":"4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1","Type":"ContainerStarted","Data":"d252f23fd13373ac50c6da5496f8590e58c70be432b2cb20d1d09356ff4ef68e"} Jul 21 18:45:59.591551 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:59.591524 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458843696" Jul 21 18:45:59.611410 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:59.611381 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7562-140036458843696_4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1/test-curl-7562-140036458843696/0.log" Jul 21 18:45:59.636831 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:59.636797 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7562-140036458843696"] Jul 21 18:45:59.642783 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:59.642754 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7562-140036458843696"] Jul 21 18:45:59.767316 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:59.767228 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwqnb\" (UniqueName: \"kubernetes.io/projected/4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1-kube-api-access-vwqnb\") pod \"4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1\" (UID: \"4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1\") " Jul 21 18:45:59.769408 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:59.769380 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1-kube-api-access-vwqnb" (OuterVolumeSpecName: "kube-api-access-vwqnb") pod "4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1" (UID: "4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1"). InnerVolumeSpecName "kube-api-access-vwqnb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:45:59.868800 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:45:59.868761 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwqnb\" (UniqueName: \"kubernetes.io/projected/4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1-kube-api-access-vwqnb\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:46:00.089524 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:00.089441 2568 scope.go:117] "RemoveContainer" containerID="bc822c7ffdeb609944898c887e8081fd0215ee0e8b66f4423418e70e6d7ba6f4" Jul 21 18:46:00.098026 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:00.098002 2568 scope.go:117] "RemoveContainer" containerID="12c1b6d19af58729f1312c349275fa5630f7c24b6209db4ea2f31be2a3e6e7ab" Jul 21 18:46:00.462344 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:00.462315 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458843696" Jul 21 18:46:00.462552 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:00.462368 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d252f23fd13373ac50c6da5496f8590e58c70be432b2cb20d1d09356ff4ef68e" Jul 21 18:46:00.472114 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:46:00.472080 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7562-140036458843696\" is forbidden: User \"system:node:ip-10-0-136-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-17.ec2.internal' and this object" podUID="4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1" pod="opendatahub/test-curl-7562-140036458843696" Jul 21 18:46:01.218899 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:01.218862 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1" path="/var/lib/kubelet/pods/4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1/volumes" Jul 21 18:46:01.430913 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:01.430875 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:46:03.076063 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:03.076029 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7562-140036458791968"] Jul 21 18:46:03.076553 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:03.076383 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1" containerName="test-curl-7562-140036458843696" Jul 21 18:46:03.076553 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:03.076394 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1" containerName="test-curl-7562-140036458843696" Jul 21 18:46:03.076553 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:03.076454 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ba518d5-5930-4ee4-a3fe-b7c65cb19ab1" containerName="test-curl-7562-140036458843696" Jul 21 18:46:03.079658 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:03.079637 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458791968" Jul 21 18:46:03.082116 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:03.082090 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lfsn8\"" Jul 21 18:46:03.084284 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:03.084255 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7562-140036458791968"] Jul 21 18:46:03.197096 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:03.197054 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wrdv\" (UniqueName: \"kubernetes.io/projected/6441ee3f-6a9a-4a67-b981-0ded33224a2c-kube-api-access-8wrdv\") pod \"test-curl-7562-140036458791968\" (UID: \"6441ee3f-6a9a-4a67-b981-0ded33224a2c\") " pod="opendatahub/test-curl-7562-140036458791968" Jul 21 18:46:03.298464 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:03.298422 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wrdv\" (UniqueName: \"kubernetes.io/projected/6441ee3f-6a9a-4a67-b981-0ded33224a2c-kube-api-access-8wrdv\") pod \"test-curl-7562-140036458791968\" (UID: \"6441ee3f-6a9a-4a67-b981-0ded33224a2c\") " pod="opendatahub/test-curl-7562-140036458791968" Jul 21 18:46:03.306812 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:03.306769 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wrdv\" (UniqueName: \"kubernetes.io/projected/6441ee3f-6a9a-4a67-b981-0ded33224a2c-kube-api-access-8wrdv\") pod \"test-curl-7562-140036458791968\" (UID: \"6441ee3f-6a9a-4a67-b981-0ded33224a2c\") " pod="opendatahub/test-curl-7562-140036458791968" Jul 21 18:46:03.390494 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:03.390440 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458791968" Jul 21 18:46:03.524014 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:03.523989 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7562-140036458791968"] Jul 21 18:46:03.526053 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:46:03.526017 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6441ee3f_6a9a_4a67_b981_0ded33224a2c.slice/crio-7e822520010fce9383252a2e66f865200c704506d576fbf19515a34ec01a5210 WatchSource:0}: Error finding container 7e822520010fce9383252a2e66f865200c704506d576fbf19515a34ec01a5210: Status 404 returned error can't find the container with id 7e822520010fce9383252a2e66f865200c704506d576fbf19515a34ec01a5210 Jul 21 18:46:04.488143 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:04.488107 2568 generic.go:356] "Generic (PLEG): container finished" podID="6441ee3f-6a9a-4a67-b981-0ded33224a2c" containerID="df1c3a1eb7e7fba4b4e999959e020e9dac4fb78ecd696169c71c6d15f3d18632" exitCode=0 Jul 21 18:46:04.488516 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:04.488168 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7562-140036458791968" event={"ID":"6441ee3f-6a9a-4a67-b981-0ded33224a2c","Type":"ContainerDied","Data":"df1c3a1eb7e7fba4b4e999959e020e9dac4fb78ecd696169c71c6d15f3d18632"} Jul 21 18:46:04.488516 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:04.488200 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7562-140036458791968" event={"ID":"6441ee3f-6a9a-4a67-b981-0ded33224a2c","Type":"ContainerStarted","Data":"7e822520010fce9383252a2e66f865200c704506d576fbf19515a34ec01a5210"} Jul 21 18:46:05.637652 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:05.637621 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458791968" Jul 21 18:46:05.657813 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:05.657786 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7562-140036458791968_6441ee3f-6a9a-4a67-b981-0ded33224a2c/test-curl-7562-140036458791968/0.log" Jul 21 18:46:05.683947 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:05.683897 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7562-140036458791968"] Jul 21 18:46:05.687184 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:05.687158 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7562-140036458791968"] Jul 21 18:46:05.804501 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:05.804422 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7562-140036458791968"] Jul 21 18:46:05.804783 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:05.804769 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6441ee3f-6a9a-4a67-b981-0ded33224a2c" containerName="test-curl-7562-140036458791968" Jul 21 18:46:05.804825 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:05.804784 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6441ee3f-6a9a-4a67-b981-0ded33224a2c" containerName="test-curl-7562-140036458791968" Jul 21 18:46:05.804868 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:05.804860 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6441ee3f-6a9a-4a67-b981-0ded33224a2c" containerName="test-curl-7562-140036458791968" Jul 21 18:46:05.807807 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:05.807789 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458791968" Jul 21 18:46:05.814816 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:05.814784 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7562-140036458791968"] Jul 21 18:46:05.821659 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:05.821626 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wrdv\" (UniqueName: \"kubernetes.io/projected/6441ee3f-6a9a-4a67-b981-0ded33224a2c-kube-api-access-8wrdv\") pod \"6441ee3f-6a9a-4a67-b981-0ded33224a2c\" (UID: \"6441ee3f-6a9a-4a67-b981-0ded33224a2c\") " Jul 21 18:46:05.823908 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:05.823868 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6441ee3f-6a9a-4a67-b981-0ded33224a2c-kube-api-access-8wrdv" (OuterVolumeSpecName: "kube-api-access-8wrdv") pod "6441ee3f-6a9a-4a67-b981-0ded33224a2c" (UID: "6441ee3f-6a9a-4a67-b981-0ded33224a2c"). InnerVolumeSpecName "kube-api-access-8wrdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:46:05.922575 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:05.922535 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fslc\" (UniqueName: \"kubernetes.io/projected/bd8e8400-30de-4800-b1c2-660226addab1-kube-api-access-6fslc\") pod \"test-curl-7562-140036458791968\" (UID: \"bd8e8400-30de-4800-b1c2-660226addab1\") " pod="opendatahub/test-curl-7562-140036458791968" Jul 21 18:46:05.922758 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:05.922651 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wrdv\" (UniqueName: \"kubernetes.io/projected/6441ee3f-6a9a-4a67-b981-0ded33224a2c-kube-api-access-8wrdv\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:46:06.024090 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:06.024048 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fslc\" (UniqueName: \"kubernetes.io/projected/bd8e8400-30de-4800-b1c2-660226addab1-kube-api-access-6fslc\") pod \"test-curl-7562-140036458791968\" (UID: \"bd8e8400-30de-4800-b1c2-660226addab1\") " pod="opendatahub/test-curl-7562-140036458791968" Jul 21 18:46:06.031576 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:06.031544 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fslc\" (UniqueName: \"kubernetes.io/projected/bd8e8400-30de-4800-b1c2-660226addab1-kube-api-access-6fslc\") pod \"test-curl-7562-140036458791968\" (UID: \"bd8e8400-30de-4800-b1c2-660226addab1\") " pod="opendatahub/test-curl-7562-140036458791968" Jul 21 18:46:06.118498 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:06.118459 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458791968" Jul 21 18:46:06.259210 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:06.259184 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7562-140036458791968"] Jul 21 18:46:06.261056 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:46:06.261021 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd8e8400_30de_4800_b1c2_660226addab1.slice/crio-80091e83a32324a4225e9a1ab1bfb21dbf31d6c7d668d2e6bf242dd881f0ab3c WatchSource:0}: Error finding container 80091e83a32324a4225e9a1ab1bfb21dbf31d6c7d668d2e6bf242dd881f0ab3c: Status 404 returned error can't find the container with id 80091e83a32324a4225e9a1ab1bfb21dbf31d6c7d668d2e6bf242dd881f0ab3c Jul 21 18:46:06.500518 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:06.500475 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7562-140036458791968" event={"ID":"bd8e8400-30de-4800-b1c2-660226addab1","Type":"ContainerStarted","Data":"80091e83a32324a4225e9a1ab1bfb21dbf31d6c7d668d2e6bf242dd881f0ab3c"} Jul 21 18:46:06.501628 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:06.501609 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e822520010fce9383252a2e66f865200c704506d576fbf19515a34ec01a5210" Jul 21 18:46:06.501759 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:06.501659 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458791968" Jul 21 18:46:06.514697 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:06.514662 2568 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7562-140036458791968" oldPodUID="6441ee3f-6a9a-4a67-b981-0ded33224a2c" podUID="bd8e8400-30de-4800-b1c2-660226addab1" Jul 21 18:46:07.219202 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:07.219156 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6441ee3f-6a9a-4a67-b981-0ded33224a2c" path="/var/lib/kubelet/pods/6441ee3f-6a9a-4a67-b981-0ded33224a2c/volumes" Jul 21 18:46:07.508538 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:07.508449 2568 generic.go:356] "Generic (PLEG): container finished" podID="bd8e8400-30de-4800-b1c2-660226addab1" containerID="72e3f8add570cdac814056af4ee67ef948ad8f845a8b31774eae7b1297c1eb37" exitCode=0 Jul 21 18:46:07.508688 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:07.508549 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7562-140036458791968" event={"ID":"bd8e8400-30de-4800-b1c2-660226addab1","Type":"ContainerDied","Data":"72e3f8add570cdac814056af4ee67ef948ad8f845a8b31774eae7b1297c1eb37"} Jul 21 18:46:08.648919 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:08.648894 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458791968" Jul 21 18:46:08.672373 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:08.672348 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7562-140036458791968_bd8e8400-30de-4800-b1c2-660226addab1/test-curl-7562-140036458791968/0.log" Jul 21 18:46:08.699423 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:08.699392 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7562-140036458791968"] Jul 21 18:46:08.704657 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:08.704633 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7562-140036458791968"] Jul 21 18:46:08.748479 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:08.748449 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fslc\" (UniqueName: \"kubernetes.io/projected/bd8e8400-30de-4800-b1c2-660226addab1-kube-api-access-6fslc\") pod \"bd8e8400-30de-4800-b1c2-660226addab1\" (UID: \"bd8e8400-30de-4800-b1c2-660226addab1\") " Jul 21 18:46:08.750449 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:08.750419 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd8e8400-30de-4800-b1c2-660226addab1-kube-api-access-6fslc" (OuterVolumeSpecName: "kube-api-access-6fslc") pod "bd8e8400-30de-4800-b1c2-660226addab1" (UID: "bd8e8400-30de-4800-b1c2-660226addab1"). InnerVolumeSpecName "kube-api-access-6fslc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:46:08.849955 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:08.849903 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fslc\" (UniqueName: \"kubernetes.io/projected/bd8e8400-30de-4800-b1c2-660226addab1-kube-api-access-6fslc\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:46:09.219835 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:09.219757 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd8e8400-30de-4800-b1c2-660226addab1" path="/var/lib/kubelet/pods/bd8e8400-30de-4800-b1c2-660226addab1/volumes" Jul 21 18:46:09.520525 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:09.520446 2568 scope.go:117] "RemoveContainer" containerID="72e3f8add570cdac814056af4ee67ef948ad8f845a8b31774eae7b1297c1eb37" Jul 21 18:46:09.520674 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:09.520445 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458791968" Jul 21 18:46:15.178911 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:15.178867 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm"] Jul 21 18:46:15.179344 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:15.179238 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8e8400-30de-4800-b1c2-660226addab1" containerName="test-curl-7562-140036458791968" Jul 21 18:46:15.179344 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:15.179251 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8e8400-30de-4800-b1c2-660226addab1" containerName="test-curl-7562-140036458791968" Jul 21 18:46:15.179344 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:15.179322 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8e8400-30de-4800-b1c2-660226addab1" containerName="test-curl-7562-140036458791968" Jul 21 18:46:15.184533 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:15.184511 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm" Jul 21 18:46:15.187537 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:15.187503 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:46:15.190136 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:15.190109 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm"] Jul 21 18:46:15.197272 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:15.197240 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9bf56ba-8da3-4891-aa6e-0b50fdd95bae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm\" (UID: \"c9bf56ba-8da3-4891-aa6e-0b50fdd95bae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm" Jul 21 18:46:15.297856 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:15.297798 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9bf56ba-8da3-4891-aa6e-0b50fdd95bae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm\" (UID: \"c9bf56ba-8da3-4891-aa6e-0b50fdd95bae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm" Jul 21 18:46:15.298466 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:15.298445 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9bf56ba-8da3-4891-aa6e-0b50fdd95bae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm\" (UID: \"c9bf56ba-8da3-4891-aa6e-0b50fdd95bae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm" Jul 21 18:46:15.497850 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:15.497765 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm" Jul 21 18:46:15.633922 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:15.633893 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm"] Jul 21 18:46:15.635679 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:46:15.635651 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9bf56ba_8da3_4891_aa6e_0b50fdd95bae.slice/crio-e1c8db74d04182eaecce2c8c20dee344ca5897895b15f5a03157ae7b441258ed WatchSource:0}: Error finding container e1c8db74d04182eaecce2c8c20dee344ca5897895b15f5a03157ae7b441258ed: Status 404 returned error can't find the container with id e1c8db74d04182eaecce2c8c20dee344ca5897895b15f5a03157ae7b441258ed Jul 21 18:46:15.787545 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:15.787513 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:15.887072 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:15.887043 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:16.561190 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:16.561153 2568 generic.go:356] "Generic (PLEG): container finished" podID="c9bf56ba-8da3-4891-aa6e-0b50fdd95bae" containerID="c16f51497ea71d6c999897f641d53cd76965bdd6defbd5cc4862f6f3d6f42480" exitCode=0 Jul 21 18:46:16.561531 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:16.561230 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm" event={"ID":"c9bf56ba-8da3-4891-aa6e-0b50fdd95bae","Type":"ContainerDied","Data":"c16f51497ea71d6c999897f641d53cd76965bdd6defbd5cc4862f6f3d6f42480"} Jul 21 18:46:16.561531 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:16.561282 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm" event={"ID":"c9bf56ba-8da3-4891-aa6e-0b50fdd95bae","Type":"ContainerStarted","Data":"e1c8db74d04182eaecce2c8c20dee344ca5897895b15f5a03157ae7b441258ed"} Jul 21 18:46:17.695386 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:17.695358 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm" Jul 21 18:46:17.715552 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:17.715525 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9bf56ba-8da3-4891-aa6e-0b50fdd95bae-maas-api-service-ca\") pod \"c9bf56ba-8da3-4891-aa6e-0b50fdd95bae\" (UID: \"c9bf56ba-8da3-4891-aa6e-0b50fdd95bae\") " Jul 21 18:46:17.715852 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:17.715830 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9bf56ba-8da3-4891-aa6e-0b50fdd95bae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c9bf56ba-8da3-4891-aa6e-0b50fdd95bae" (UID: "c9bf56ba-8da3-4891-aa6e-0b50fdd95bae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:46:17.816886 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:17.816791 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9bf56ba-8da3-4891-aa6e-0b50fdd95bae-maas-api-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:46:18.573333 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:18.573302 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm" Jul 21 18:46:18.573504 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:18.573297 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm" event={"ID":"c9bf56ba-8da3-4891-aa6e-0b50fdd95bae","Type":"ContainerDied","Data":"e1c8db74d04182eaecce2c8c20dee344ca5897895b15f5a03157ae7b441258ed"} Jul 21 18:46:18.573504 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:18.573404 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1c8db74d04182eaecce2c8c20dee344ca5897895b15f5a03157ae7b441258ed" Jul 21 18:46:25.726301 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:25.726218 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g"] Jul 21 18:46:25.726712 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:25.726425 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" podUID="a97926db-d3fd-4560-93e4-2f3b346cb374" containerName="maas-api" containerID="cri-o://073e22cc95a02c28d18f664c5707cedd434ed72585bcff83d30f1fa498c9172e" gracePeriod=30 Jul 21 18:46:25.751967 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:25.751922 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c1a1f4-29744325-bcwwf"] Jul 21 18:46:25.757972 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:25.757931 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c1a1f4-29744325-bcwwf"] Jul 21 18:46:25.759989 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:25.759966 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:25.976153 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:25.976128 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:46:26.085553 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.085470 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxh67\" (UniqueName: \"kubernetes.io/projected/a97926db-d3fd-4560-93e4-2f3b346cb374-kube-api-access-hxh67\") pod \"a97926db-d3fd-4560-93e4-2f3b346cb374\" (UID: \"a97926db-d3fd-4560-93e4-2f3b346cb374\") " Jul 21 18:46:26.085553 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.085523 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a97926db-d3fd-4560-93e4-2f3b346cb374-maas-api-tls\") pod \"a97926db-d3fd-4560-93e4-2f3b346cb374\" (UID: \"a97926db-d3fd-4560-93e4-2f3b346cb374\") " Jul 21 18:46:26.085738 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.085565 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97926db-d3fd-4560-93e4-2f3b346cb374-openshift-service-ca\") pod \"a97926db-d3fd-4560-93e4-2f3b346cb374\" (UID: \"a97926db-d3fd-4560-93e4-2f3b346cb374\") " Jul 21 18:46:26.086012 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.085988 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a97926db-d3fd-4560-93e4-2f3b346cb374-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a97926db-d3fd-4560-93e4-2f3b346cb374" (UID: "a97926db-d3fd-4560-93e4-2f3b346cb374"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:46:26.087625 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.087603 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a97926db-d3fd-4560-93e4-2f3b346cb374-kube-api-access-hxh67" (OuterVolumeSpecName: "kube-api-access-hxh67") pod "a97926db-d3fd-4560-93e4-2f3b346cb374" (UID: "a97926db-d3fd-4560-93e4-2f3b346cb374"). InnerVolumeSpecName "kube-api-access-hxh67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:46:26.087708 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.087653 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a97926db-d3fd-4560-93e4-2f3b346cb374-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a97926db-d3fd-4560-93e4-2f3b346cb374" (UID: "a97926db-d3fd-4560-93e4-2f3b346cb374"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:46:26.187058 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.187020 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxh67\" (UniqueName: \"kubernetes.io/projected/a97926db-d3fd-4560-93e4-2f3b346cb374-kube-api-access-hxh67\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:46:26.187058 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.187054 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a97926db-d3fd-4560-93e4-2f3b346cb374-maas-api-tls\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:46:26.187058 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.187063 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97926db-d3fd-4560-93e4-2f3b346cb374-openshift-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:46:26.620194 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.620155 2568 generic.go:356] "Generic (PLEG): container finished" podID="a97926db-d3fd-4560-93e4-2f3b346cb374" containerID="073e22cc95a02c28d18f664c5707cedd434ed72585bcff83d30f1fa498c9172e" exitCode=0 Jul 21 18:46:26.620373 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.620198 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" Jul 21 18:46:26.620373 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.620242 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" event={"ID":"a97926db-d3fd-4560-93e4-2f3b346cb374","Type":"ContainerDied","Data":"073e22cc95a02c28d18f664c5707cedd434ed72585bcff83d30f1fa498c9172e"} Jul 21 18:46:26.620373 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.620295 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g" event={"ID":"a97926db-d3fd-4560-93e4-2f3b346cb374","Type":"ContainerDied","Data":"4b5a6ddb45f9e66dece1a0484334495a16c7412d8abb6034171b763ef534e52f"} Jul 21 18:46:26.620373 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.620322 2568 scope.go:117] "RemoveContainer" containerID="073e22cc95a02c28d18f664c5707cedd434ed72585bcff83d30f1fa498c9172e" Jul 21 18:46:26.629573 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.629551 2568 scope.go:117] "RemoveContainer" containerID="073e22cc95a02c28d18f664c5707cedd434ed72585bcff83d30f1fa498c9172e" Jul 21 18:46:26.629846 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:46:26.629827 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"073e22cc95a02c28d18f664c5707cedd434ed72585bcff83d30f1fa498c9172e\": container with ID starting with 073e22cc95a02c28d18f664c5707cedd434ed72585bcff83d30f1fa498c9172e not found: ID does not exist" containerID="073e22cc95a02c28d18f664c5707cedd434ed72585bcff83d30f1fa498c9172e" Jul 21 18:46:26.629895 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.629855 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"073e22cc95a02c28d18f664c5707cedd434ed72585bcff83d30f1fa498c9172e"} err="failed to get container status \"073e22cc95a02c28d18f664c5707cedd434ed72585bcff83d30f1fa498c9172e\": rpc error: code = NotFound desc = could not find container \"073e22cc95a02c28d18f664c5707cedd434ed72585bcff83d30f1fa498c9172e\": container with ID starting with 073e22cc95a02c28d18f664c5707cedd434ed72585bcff83d30f1fa498c9172e not found: ID does not exist" Jul 21 18:46:26.640872 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.640839 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g"] Jul 21 18:46:26.646904 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:26.646883 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c1a1f4-7c9fdb4b49-c9d2g"] Jul 21 18:46:27.218820 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:27.218789 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e919a63-fb1b-4de6-a768-c0fbbe0267ca" path="/var/lib/kubelet/pods/7e919a63-fb1b-4de6-a768-c0fbbe0267ca/volumes" Jul 21 18:46:27.219202 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:27.219154 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a97926db-d3fd-4560-93e4-2f3b346cb374" path="/var/lib/kubelet/pods/a97926db-d3fd-4560-93e4-2f3b346cb374/volumes" Jul 21 18:46:28.528906 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:28.528870 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:46:30.261952 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:30.261904 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:30.275715 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:30.275679 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm"] Jul 21 18:46:30.278856 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:30.278825 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c1a1f4-q57wm"] Jul 21 18:46:30.792239 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:30.792208 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:31.218904 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:31.218868 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9bf56ba-8da3-4891-aa6e-0b50fdd95bae" path="/var/lib/kubelet/pods/c9bf56ba-8da3-4891-aa6e-0b50fdd95bae/volumes" Jul 21 18:46:36.717047 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:36.717013 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:36.838258 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:36.838225 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:37.011331 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.011245 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:37.363668 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.363628 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt"] Jul 21 18:46:37.364067 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.364050 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9bf56ba-8da3-4891-aa6e-0b50fdd95bae" containerName="revoke-keys" Jul 21 18:46:37.364169 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.364069 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9bf56ba-8da3-4891-aa6e-0b50fdd95bae" containerName="revoke-keys" Jul 21 18:46:37.364169 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.364088 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a97926db-d3fd-4560-93e4-2f3b346cb374" containerName="maas-api" Jul 21 18:46:37.364169 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.364096 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a97926db-d3fd-4560-93e4-2f3b346cb374" containerName="maas-api" Jul 21 18:46:37.364316 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.364184 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9bf56ba-8da3-4891-aa6e-0b50fdd95bae" containerName="revoke-keys" Jul 21 18:46:37.364316 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.364201 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a97926db-d3fd-4560-93e4-2f3b346cb374" containerName="maas-api" Jul 21 18:46:37.368868 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.368836 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt" Jul 21 18:46:37.371354 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.371324 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5zqnc\"" Jul 21 18:46:37.374792 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.374703 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt"] Jul 21 18:46:37.433550 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.433495 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:37.487871 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.487831 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/839a050e-723e-4045-a90d-5d2c4d421a0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt\" (UID: \"839a050e-723e-4045-a90d-5d2c4d421a0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt" Jul 21 18:46:37.589105 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.589062 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/839a050e-723e-4045-a90d-5d2c4d421a0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt\" (UID: \"839a050e-723e-4045-a90d-5d2c4d421a0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt" Jul 21 18:46:37.589709 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.589689 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/839a050e-723e-4045-a90d-5d2c4d421a0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt\" (UID: \"839a050e-723e-4045-a90d-5d2c4d421a0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt" Jul 21 18:46:37.683537 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.683447 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt" Jul 21 18:46:37.821624 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.821600 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt"] Jul 21 18:46:37.823801 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:46:37.823772 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod839a050e_723e_4045_a90d_5d2c4d421a0c.slice/crio-fba9dfc1cf15c2deda8c29d5544ab34b69e31f339fb50c85b29b1c299c13fb48 WatchSource:0}: Error finding container fba9dfc1cf15c2deda8c29d5544ab34b69e31f339fb50c85b29b1c299c13fb48: Status 404 returned error can't find the container with id fba9dfc1cf15c2deda8c29d5544ab34b69e31f339fb50c85b29b1c299c13fb48 Jul 21 18:46:37.905996 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.905963 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:38.000041 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:37.999960 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:38.686322 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:38.686292 2568 generic.go:356] "Generic (PLEG): container finished" podID="839a050e-723e-4045-a90d-5d2c4d421a0c" containerID="f009826a513b032437cb6906022e2418accb5002aedaff825b58b5c63a383bfa" exitCode=0 Jul 21 18:46:38.686502 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:38.686328 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt" event={"ID":"839a050e-723e-4045-a90d-5d2c4d421a0c","Type":"ContainerDied","Data":"f009826a513b032437cb6906022e2418accb5002aedaff825b58b5c63a383bfa"} Jul 21 18:46:38.686502 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:38.686373 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt" event={"ID":"839a050e-723e-4045-a90d-5d2c4d421a0c","Type":"ContainerStarted","Data":"fba9dfc1cf15c2deda8c29d5544ab34b69e31f339fb50c85b29b1c299c13fb48"} Jul 21 18:46:39.819948 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:39.819916 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt" Jul 21 18:46:39.909724 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:39.909683 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/839a050e-723e-4045-a90d-5d2c4d421a0c-maas-api-service-ca\") pod \"839a050e-723e-4045-a90d-5d2c4d421a0c\" (UID: \"839a050e-723e-4045-a90d-5d2c4d421a0c\") " Jul 21 18:46:39.910129 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:39.910104 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/839a050e-723e-4045-a90d-5d2c4d421a0c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "839a050e-723e-4045-a90d-5d2c4d421a0c" (UID: "839a050e-723e-4045-a90d-5d2c4d421a0c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:46:40.011374 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:40.011282 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/839a050e-723e-4045-a90d-5d2c4d421a0c-maas-api-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:46:40.699264 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:40.699235 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt" Jul 21 18:46:40.699264 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:40.699251 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt" event={"ID":"839a050e-723e-4045-a90d-5d2c4d421a0c","Type":"ContainerDied","Data":"fba9dfc1cf15c2deda8c29d5544ab34b69e31f339fb50c85b29b1c299c13fb48"} Jul 21 18:46:40.699517 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:40.699291 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fba9dfc1cf15c2deda8c29d5544ab34b69e31f339fb50c85b29b1c299c13fb48" Jul 21 18:46:47.917916 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:47.917884 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm"] Jul 21 18:46:47.918329 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:47.918127 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" podUID="ca764bb0-54dc-4855-a53a-0f55799886bc" containerName="maas-api" containerID="cri-o://3e5e94283adf08c7e6f043114138d25aa6c9af6d3aa3a612bf846163da32bc40" gracePeriod=30 Jul 21 18:46:47.935649 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:47.935620 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0be774-29744325-bg8fs"] Jul 21 18:46:47.941300 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:47.941271 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:47.941523 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:47.941501 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0be774-29744325-bg8fs"] Jul 21 18:46:48.164781 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.164749 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:46:48.281913 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.281828 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca764bb0-54dc-4855-a53a-0f55799886bc-openshift-service-ca\") pod \"ca764bb0-54dc-4855-a53a-0f55799886bc\" (UID: \"ca764bb0-54dc-4855-a53a-0f55799886bc\") " Jul 21 18:46:48.281913 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.281878 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsrqs\" (UniqueName: \"kubernetes.io/projected/ca764bb0-54dc-4855-a53a-0f55799886bc-kube-api-access-wsrqs\") pod \"ca764bb0-54dc-4855-a53a-0f55799886bc\" (UID: \"ca764bb0-54dc-4855-a53a-0f55799886bc\") " Jul 21 18:46:48.282109 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.281980 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca764bb0-54dc-4855-a53a-0f55799886bc-maas-api-tls\") pod \"ca764bb0-54dc-4855-a53a-0f55799886bc\" (UID: \"ca764bb0-54dc-4855-a53a-0f55799886bc\") " Jul 21 18:46:48.282348 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.282312 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca764bb0-54dc-4855-a53a-0f55799886bc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca764bb0-54dc-4855-a53a-0f55799886bc" (UID: "ca764bb0-54dc-4855-a53a-0f55799886bc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 18:46:48.284124 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.284099 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca764bb0-54dc-4855-a53a-0f55799886bc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca764bb0-54dc-4855-a53a-0f55799886bc" (UID: "ca764bb0-54dc-4855-a53a-0f55799886bc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 18:46:48.284212 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.284143 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca764bb0-54dc-4855-a53a-0f55799886bc-kube-api-access-wsrqs" (OuterVolumeSpecName: "kube-api-access-wsrqs") pod "ca764bb0-54dc-4855-a53a-0f55799886bc" (UID: "ca764bb0-54dc-4855-a53a-0f55799886bc"). InnerVolumeSpecName "kube-api-access-wsrqs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 18:46:48.383421 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.383384 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca764bb0-54dc-4855-a53a-0f55799886bc-maas-api-tls\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:46:48.383421 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.383417 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca764bb0-54dc-4855-a53a-0f55799886bc-openshift-service-ca\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:46:48.383421 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.383427 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsrqs\" (UniqueName: \"kubernetes.io/projected/ca764bb0-54dc-4855-a53a-0f55799886bc-kube-api-access-wsrqs\") on node \"ip-10-0-136-17.ec2.internal\" DevicePath \"\"" Jul 21 18:46:48.746121 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.746084 2568 generic.go:356] "Generic (PLEG): container finished" podID="ca764bb0-54dc-4855-a53a-0f55799886bc" containerID="3e5e94283adf08c7e6f043114138d25aa6c9af6d3aa3a612bf846163da32bc40" exitCode=0 Jul 21 18:46:48.746297 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.746129 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" Jul 21 18:46:48.746297 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.746173 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" event={"ID":"ca764bb0-54dc-4855-a53a-0f55799886bc","Type":"ContainerDied","Data":"3e5e94283adf08c7e6f043114138d25aa6c9af6d3aa3a612bf846163da32bc40"} Jul 21 18:46:48.746297 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.746231 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm" event={"ID":"ca764bb0-54dc-4855-a53a-0f55799886bc","Type":"ContainerDied","Data":"a010b5a27b0f9399078dea37dc71c9e65b743b921bec8a8f4365bfbd5bc4a68f"} Jul 21 18:46:48.746297 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.746259 2568 scope.go:117] "RemoveContainer" containerID="3e5e94283adf08c7e6f043114138d25aa6c9af6d3aa3a612bf846163da32bc40" Jul 21 18:46:48.761509 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.761483 2568 scope.go:117] "RemoveContainer" containerID="3e5e94283adf08c7e6f043114138d25aa6c9af6d3aa3a612bf846163da32bc40" Jul 21 18:46:48.761898 ip-10-0-136-17 kubenswrapper[2568]: E0721 18:46:48.761873 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e5e94283adf08c7e6f043114138d25aa6c9af6d3aa3a612bf846163da32bc40\": container with ID starting with 3e5e94283adf08c7e6f043114138d25aa6c9af6d3aa3a612bf846163da32bc40 not found: ID does not exist" containerID="3e5e94283adf08c7e6f043114138d25aa6c9af6d3aa3a612bf846163da32bc40" Jul 21 18:46:48.762010 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.761910 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e5e94283adf08c7e6f043114138d25aa6c9af6d3aa3a612bf846163da32bc40"} err="failed to get container status \"3e5e94283adf08c7e6f043114138d25aa6c9af6d3aa3a612bf846163da32bc40\": rpc error: code = NotFound desc = could not find container \"3e5e94283adf08c7e6f043114138d25aa6c9af6d3aa3a612bf846163da32bc40\": container with ID starting with 3e5e94283adf08c7e6f043114138d25aa6c9af6d3aa3a612bf846163da32bc40 not found: ID does not exist" Jul 21 18:46:48.774668 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.774636 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm"] Jul 21 18:46:48.778119 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:48.778093 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0be774-646487775f-6qnmm"] Jul 21 18:46:49.218889 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:49.218852 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca764bb0-54dc-4855-a53a-0f55799886bc" path="/var/lib/kubelet/pods/ca764bb0-54dc-4855-a53a-0f55799886bc/volumes" Jul 21 18:46:49.219282 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:49.219180 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff353bf9-4ff4-4af0-92d6-ebe5eb8a782b" path="/var/lib/kubelet/pods/ff353bf9-4ff4-4af0-92d6-ebe5eb8a782b/volumes" Jul 21 18:46:51.024921 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:51.024874 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-49bb6"] Jul 21 18:46:52.443785 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:52.443745 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt"] Jul 21 18:46:52.445760 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:52.445738 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0be774-jdpqt"] Jul 21 18:46:52.447673 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:52.447649 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:52.827586 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:52.827496 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:53.218655 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:53.218612 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="839a050e-723e-4045-a90d-5d2c4d421a0c" path="/var/lib/kubelet/pods/839a050e-723e-4045-a90d-5d2c4d421a0c/volumes" Jul 21 18:46:58.906691 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:58.906657 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:58.973198 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:58.973167 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:59.199343 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:59.199254 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:46:59.520884 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:46:59.520778 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:47:06.008822 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:06.008795 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b54c48d7b-n92j9_1cdfb9a3-3beb-4685-817e-f17651d1fb1c/copy-manifests/0.log" Jul 21 18:47:06.022051 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:06.022022 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b54c48d7b-n92j9_1cdfb9a3-3beb-4685-817e-f17651d1fb1c/manager/0.log" Jul 21 18:47:06.740889 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:06.740858 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744295-9rh92_27af902b-18bc-49f4-9721-a481d4ae273f/cleanup/0.log" Jul 21 18:47:06.856383 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:06.856348 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744310-btlwd_a1aa9a8b-932f-4bc6-a9c0-abdec2c2e440/cleanup/0.log" Jul 21 18:47:06.970584 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:06.970552 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744325-kkhxd_c1a1b537-404d-4c3a-9229-a03f1191eb29/cleanup/0.log" Jul 21 18:47:07.895184 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:07.895152 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz_b1023893-fbc7-47d0-a023-a9f0a76b6ccd/pull/0.log" Jul 21 18:47:07.902540 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:07.902498 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz_b1023893-fbc7-47d0-a023-a9f0a76b6ccd/extract/0.log" Jul 21 18:47:07.909418 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:07.909396 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz_b1023893-fbc7-47d0-a023-a9f0a76b6ccd/util/0.log" Jul 21 18:47:08.029224 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:08.029187 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt_2f68f4f1-d977-4296-a62e-e9fda2a9fd59/util/0.log" Jul 21 18:47:08.036666 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:08.036638 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt_2f68f4f1-d977-4296-a62e-e9fda2a9fd59/pull/0.log" Jul 21 18:47:08.043439 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:08.043409 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt_2f68f4f1-d977-4296-a62e-e9fda2a9fd59/extract/0.log" Jul 21 18:47:08.171710 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:08.171628 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx_0d401d4c-07a8-42c7-afeb-44ee744e346e/util/0.log" Jul 21 18:47:08.179095 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:08.179070 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx_0d401d4c-07a8-42c7-afeb-44ee744e346e/pull/0.log" Jul 21 18:47:08.186108 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:08.186089 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx_0d401d4c-07a8-42c7-afeb-44ee744e346e/extract/0.log" Jul 21 18:47:08.311336 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:08.311310 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn_94cf9b0f-b8bc-466b-91af-2cb153917fb4/util/0.log" Jul 21 18:47:08.320253 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:08.320229 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn_94cf9b0f-b8bc-466b-91af-2cb153917fb4/pull/0.log" Jul 21 18:47:08.326286 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:08.326270 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn_94cf9b0f-b8bc-466b-91af-2cb153917fb4/extract/0.log" Jul 21 18:47:09.131491 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:09.131458 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-49bb6_c576b06f-1bc5-4c83-b25a-45757fbbf3d5/limitador/0.log" Jul 21 18:47:09.728054 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:09.728024 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fbm4v_cd4980f8-00b8-4449-96f6-f409c3d52c89/discovery/0.log" Jul 21 18:47:10.438478 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:10.438447 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-799cfd6768-8vvdp_043e03ca-7086-49e3-96b3-b824757c652a/router/0.log" Jul 21 18:47:10.902634 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:10.902611 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-87w97_85fc0cee-7513-4376-a953-02787f9e096f/main/0.log" Jul 21 18:47:16.632415 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:16.632375 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:47:16.731258 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:16.731223 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" limit=1024 Jul 21 18:47:19.425140 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:19.425112 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hrk92_f9411eb0-f8da-4954-8df7-8fa2df70b0f0/global-pull-secret-syncer/0.log" Jul 21 18:47:19.563444 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:19.563410 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gtv75_2c224dc7-b84d-4e28-84fe-686e4f9735ac/konnectivity-agent/0.log" Jul 21 18:47:19.664847 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:19.664807 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-17.ec2.internal_0a6fe56b86f91ef9f0ca8f3090eba9bd/haproxy/0.log" Jul 21 18:47:23.605907 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:23.605860 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz_b1023893-fbc7-47d0-a023-a9f0a76b6ccd/extract/0.log" Jul 21 18:47:23.632849 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:23.632807 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz_b1023893-fbc7-47d0-a023-a9f0a76b6ccd/util/0.log" Jul 21 18:47:23.687800 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:23.687773 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592czfz_b1023893-fbc7-47d0-a023-a9f0a76b6ccd/pull/0.log" Jul 21 18:47:23.722860 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:23.722825 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt_2f68f4f1-d977-4296-a62e-e9fda2a9fd59/extract/0.log" Jul 21 18:47:23.751102 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:23.751066 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt_2f68f4f1-d977-4296-a62e-e9fda2a9fd59/util/0.log" Jul 21 18:47:23.779397 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:23.779369 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gh7zt_2f68f4f1-d977-4296-a62e-e9fda2a9fd59/pull/0.log" Jul 21 18:47:23.804858 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:23.804830 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx_0d401d4c-07a8-42c7-afeb-44ee744e346e/extract/0.log" Jul 21 18:47:23.833532 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:23.833506 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx_0d401d4c-07a8-42c7-afeb-44ee744e346e/util/0.log" Jul 21 18:47:23.865460 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:23.865392 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739hhkx_0d401d4c-07a8-42c7-afeb-44ee744e346e/pull/0.log" Jul 21 18:47:23.893106 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:23.893076 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn_94cf9b0f-b8bc-466b-91af-2cb153917fb4/extract/0.log" Jul 21 18:47:23.923544 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:23.923511 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn_94cf9b0f-b8bc-466b-91af-2cb153917fb4/util/0.log" Jul 21 18:47:23.948888 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:23.948858 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1c7twn_94cf9b0f-b8bc-466b-91af-2cb153917fb4/pull/0.log" Jul 21 18:47:24.621437 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:24.621413 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-49bb6_c576b06f-1bc5-4c83-b25a-45757fbbf3d5/limitador/0.log" Jul 21 18:47:26.428466 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:26.428436 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-tvbh7_d4717f23-7fe1-4c06-b411-0bb095e21361/cluster-monitoring-operator/0.log" Jul 21 18:47:26.594927 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:26.594894 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cg5ps_f91d7dcc-7895-4302-998c-c26e296b5c85/node-exporter/0.log" Jul 21 18:47:26.615203 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:26.615177 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cg5ps_f91d7dcc-7895-4302-998c-c26e296b5c85/kube-rbac-proxy/0.log" Jul 21 18:47:26.634770 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:26.634743 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cg5ps_f91d7dcc-7895-4302-998c-c26e296b5c85/init-textfile/0.log" Jul 21 18:47:27.280322 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:27.280295 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77544fd5d-qszwk_6eac6911-3847-4b9d-ad58-220b04d1fa9d/telemeter-client/0.log" Jul 21 18:47:27.310160 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:27.310130 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77544fd5d-qszwk_6eac6911-3847-4b9d-ad58-220b04d1fa9d/reload/0.log" Jul 21 18:47:27.337043 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:27.337014 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77544fd5d-qszwk_6eac6911-3847-4b9d-ad58-220b04d1fa9d/kube-rbac-proxy/0.log" Jul 21 18:47:28.856262 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:28.856236 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-zvk8d_8d00aa82-0b34-4f8b-a7a1-3c4e3458833f/networking-console-plugin/0.log" Jul 21 18:47:29.471622 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:29.471593 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/1.log" Jul 21 18:47:29.476102 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:29.476082 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-74hdc_7bc042d5-1297-4ac5-8237-76c5e2ab7ddd/console-operator/2.log" Jul 21 18:47:30.040918 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.040889 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d8bb9f966-tcgsr_6921d7b6-2fc7-46e1-8a34-b544fa62e23d/console/0.log" Jul 21 18:47:30.080999 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.080969 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w"] Jul 21 18:47:30.081315 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.081302 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="839a050e-723e-4045-a90d-5d2c4d421a0c" containerName="revoke-keys" Jul 21 18:47:30.081358 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.081319 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="839a050e-723e-4045-a90d-5d2c4d421a0c" containerName="revoke-keys" Jul 21 18:47:30.081358 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.081348 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca764bb0-54dc-4855-a53a-0f55799886bc" containerName="maas-api" Jul 21 18:47:30.081358 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.081354 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca764bb0-54dc-4855-a53a-0f55799886bc" containerName="maas-api" Jul 21 18:47:30.081452 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.081413 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca764bb0-54dc-4855-a53a-0f55799886bc" containerName="maas-api" Jul 21 18:47:30.081452 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.081421 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="839a050e-723e-4045-a90d-5d2c4d421a0c" containerName="revoke-keys" Jul 21 18:47:30.082117 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.082100 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-xmhx6_d23f3bbd-bf10-4a67-853d-b8d2f6a875ae/download-server/0.log" Jul 21 18:47:30.084640 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.084620 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.089759 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.089737 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gg7gx\"/\"kube-root-ca.crt\"" Jul 21 18:47:30.090949 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.090664 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gg7gx\"/\"default-dockercfg-pzzcs\"" Jul 21 18:47:30.090949 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.090664 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gg7gx\"/\"openshift-service-ca.crt\"" Jul 21 18:47:30.105391 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.105357 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w"] Jul 21 18:47:30.150729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.150695 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c429abfa-b9be-4d0e-bb7c-298eabb2960a-podres\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.150729 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.150736 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c429abfa-b9be-4d0e-bb7c-298eabb2960a-lib-modules\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.150970 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.150763 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c429abfa-b9be-4d0e-bb7c-298eabb2960a-sys\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.150970 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.150849 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c429abfa-b9be-4d0e-bb7c-298eabb2960a-proc\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.150970 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.150892 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm5xw\" (UniqueName: \"kubernetes.io/projected/c429abfa-b9be-4d0e-bb7c-298eabb2960a-kube-api-access-hm5xw\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.252114 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.252079 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c429abfa-b9be-4d0e-bb7c-298eabb2960a-sys\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.252256 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.252127 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c429abfa-b9be-4d0e-bb7c-298eabb2960a-proc\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.252256 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.252157 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm5xw\" (UniqueName: \"kubernetes.io/projected/c429abfa-b9be-4d0e-bb7c-298eabb2960a-kube-api-access-hm5xw\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.252256 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.252205 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c429abfa-b9be-4d0e-bb7c-298eabb2960a-sys\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.252256 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.252209 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c429abfa-b9be-4d0e-bb7c-298eabb2960a-podres\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.252430 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.252257 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c429abfa-b9be-4d0e-bb7c-298eabb2960a-lib-modules\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.252430 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.252214 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c429abfa-b9be-4d0e-bb7c-298eabb2960a-proc\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.252430 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.252307 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c429abfa-b9be-4d0e-bb7c-298eabb2960a-podres\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.252430 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.252378 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c429abfa-b9be-4d0e-bb7c-298eabb2960a-lib-modules\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.266826 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.266796 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm5xw\" (UniqueName: \"kubernetes.io/projected/c429abfa-b9be-4d0e-bb7c-298eabb2960a-kube-api-access-hm5xw\") pod \"perf-node-gather-daemonset-qk97w\" (UID: \"c429abfa-b9be-4d0e-bb7c-298eabb2960a\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.394265 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.394216 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.522498 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.522325 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w"] Jul 21 18:47:30.525318 ip-10-0-136-17 kubenswrapper[2568]: W0721 18:47:30.525292 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc429abfa_b9be_4d0e_bb7c_298eabb2960a.slice/crio-fce346514d24667bad6b7467354bd1241d168ef8125a6ffa3f65e0145d8f6735 WatchSource:0}: Error finding container fce346514d24667bad6b7467354bd1241d168ef8125a6ffa3f65e0145d8f6735: Status 404 returned error can't find the container with id fce346514d24667bad6b7467354bd1241d168ef8125a6ffa3f65e0145d8f6735 Jul 21 18:47:30.818037 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.818011 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-6jjkh_d7074553-ba3e-43c7-8f4c-1037942bb3a9/volume-data-source-validator/0.log" Jul 21 18:47:30.976088 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.975992 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" event={"ID":"c429abfa-b9be-4d0e-bb7c-298eabb2960a","Type":"ContainerStarted","Data":"d61ace00a908169f3d4aa12a66ee2d1c3be5cd1354d4f4584169150dd7c3b1ef"} Jul 21 18:47:30.976088 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.976049 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" event={"ID":"c429abfa-b9be-4d0e-bb7c-298eabb2960a","Type":"ContainerStarted","Data":"fce346514d24667bad6b7467354bd1241d168ef8125a6ffa3f65e0145d8f6735"} Jul 21 18:47:30.976088 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.976069 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:30.991600 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:30.991536 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" podStartSLOduration=0.991520964 podStartE2EDuration="991.520964ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:47:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:47:30.9906915 +0000 UTC m=+2852.259002941" watchObservedRunningTime="2026-07-21 18:47:30.991520964 +0000 UTC m=+2852.259832404" Jul 21 18:47:31.927785 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:31.927756 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-krmfj_befe4828-08d1-4685-84ef-766f46b64135/dns/0.log" Jul 21 18:47:31.947825 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:31.947796 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-krmfj_befe4828-08d1-4685-84ef-766f46b64135/kube-rbac-proxy/0.log" Jul 21 18:47:32.072370 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:32.072341 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vlkxx_dc216396-5088-4fd3-9409-f525712b3b92/dns-node-resolver/0.log" Jul 21 18:47:32.682144 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:32.682111 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-44h8t_163da31b-27c0-4e33-b898-4a2066ccea13/node-ca/0.log" Jul 21 18:47:33.832201 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:33.832174 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fbm4v_cd4980f8-00b8-4449-96f6-f409c3d52c89/discovery/0.log" Jul 21 18:47:34.054889 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:34.054862 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-799cfd6768-8vvdp_043e03ca-7086-49e3-96b3-b824757c652a/router/0.log" Jul 21 18:47:34.777483 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:34.777454 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vvvfz_37231e4d-5ee2-44d9-9482-19501acca31d/serve-healthcheck-canary/0.log" Jul 21 18:47:35.231723 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:35.231690 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-ztxpr_80cad807-9199-42c6-8b46-8d6887025525/insights-operator/0.log" Jul 21 18:47:35.232144 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:35.232003 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-ztxpr_80cad807-9199-42c6-8b46-8d6887025525/insights-operator/1.log" Jul 21 18:47:35.252931 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:35.252905 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7gszs_afe50d97-edb0-4de3-bf15-e189d083fb5d/kube-rbac-proxy/0.log" Jul 21 18:47:35.273862 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:35.273837 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7gszs_afe50d97-edb0-4de3-bf15-e189d083fb5d/exporter/0.log" Jul 21 18:47:35.295886 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:35.295859 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7gszs_afe50d97-edb0-4de3-bf15-e189d083fb5d/extractor/0.log" Jul 21 18:47:36.991609 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:36.991582 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-qk97w" Jul 21 18:47:38.291985 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:38.291955 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b54c48d7b-n92j9_1cdfb9a3-3beb-4685-817e-f17651d1fb1c/manager/0.log" Jul 21 18:47:38.311706 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:38.311683 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b54c48d7b-n92j9_1cdfb9a3-3beb-4685-817e-f17651d1fb1c/copy-manifests/0.log" Jul 21 18:47:40.030470 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:40.030440 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fbm4v_cd4980f8-00b8-4449-96f6-f409c3d52c89/discovery/0.log" Jul 21 18:47:40.048136 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:40.048110 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-754c46d99f-947qv_89d7a857-45f8-466d-97bc-19b9235ca4fc/manager/0.log" Jul 21 18:47:40.203726 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:40.203645 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-799cfd6768-8vvdp_043e03ca-7086-49e3-96b3-b824757c652a/router/0.log" Jul 21 18:47:41.581889 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:41.581851 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fbm4v_cd4980f8-00b8-4449-96f6-f409c3d52c89/discovery/0.log" Jul 21 18:47:41.744950 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:41.744910 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-799cfd6768-8vvdp_043e03ca-7086-49e3-96b3-b824757c652a/router/0.log" Jul 21 18:47:45.042167 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:45.042133 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-swmjm_79c07e15-05b5-4984-808b-f5ed1091b13f/migrator/0.log" Jul 21 18:47:45.062339 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:45.062307 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-swmjm_79c07e15-05b5-4984-808b-f5ed1091b13f/graceful-termination/0.log" Jul 21 18:47:45.435147 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:45.435112 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-llddq_cf634bfc-df89-4e85-874a-33bd28e7eb68/kube-storage-version-migrator-operator/1.log" Jul 21 18:47:45.435835 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:45.435817 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-llddq_cf634bfc-df89-4e85-874a-33bd28e7eb68/kube-storage-version-migrator-operator/0.log" Jul 21 18:47:46.707900 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:46.707870 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q8fqf_3257df56-1fd0-4f11-a9fa-b68d6daa1a1b/kube-multus-additional-cni-plugins/0.log" Jul 21 18:47:46.728749 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:46.728721 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q8fqf_3257df56-1fd0-4f11-a9fa-b68d6daa1a1b/egress-router-binary-copy/0.log" Jul 21 18:47:46.749695 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:46.749668 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q8fqf_3257df56-1fd0-4f11-a9fa-b68d6daa1a1b/cni-plugins/0.log" Jul 21 18:47:46.770334 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:46.770300 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q8fqf_3257df56-1fd0-4f11-a9fa-b68d6daa1a1b/bond-cni-plugin/0.log" Jul 21 18:47:46.790236 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:46.790212 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q8fqf_3257df56-1fd0-4f11-a9fa-b68d6daa1a1b/routeoverride-cni/0.log" Jul 21 18:47:46.810486 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:46.810456 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q8fqf_3257df56-1fd0-4f11-a9fa-b68d6daa1a1b/whereabouts-cni-bincopy/0.log" Jul 21 18:47:46.830067 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:46.830043 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q8fqf_3257df56-1fd0-4f11-a9fa-b68d6daa1a1b/whereabouts-cni/0.log" Jul 21 18:47:46.863429 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:46.863400 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bnghg_f7cdc92e-ed55-4606-bd31-6cf648f24460/kube-multus/0.log" Jul 21 18:47:46.953727 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:46.953698 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kr8ht_89ffba65-3615-478c-856d-a75629de4606/network-metrics-daemon/0.log" Jul 21 18:47:46.972436 ip-10-0-136-17 kubenswrapper[2568]: I0721 18:47:46.972364 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kr8ht_89ffba65-3615-478c-856d-a75629de4606/kube-rbac-proxy/0.log"